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DEFINING  AND  MEASURING  THE  SUCCESS 
OF  SERVICE  CONTRACTS 


ABSTRACT 


Services  acquisition  in  the  U.S.  Department  of  Defense  (DoD)  has  continued  to  increase 
in  scope  and  dollars  in  the  past  decade.  The  DoD  has  spent  more  on  services  than  on 
supplies,  equipment  and  goods  together,  totaling  approximately  57%  of  total  acquisition 
expenditures  and  nearly  a  third  of  the  total  DoD  budget.  As  a  result,  the  agency  must 
give  greater  attention  to  the  management  of  services  acquisition.  Stakeholder  theory 
illustrates  how  acquisition  team  members  often  have  conflicting  goals  and  objectives, 
leading  to  differing  definitions  and  measurements  of  a  successful  service  contact.  We 
used  stakeholder  theory  to  address  the  following  questions:  (1)  how  are  successful  service 
contracts  within  the  DoD  being  defined  by  different  stakeholders,  (2)  how  are  service 
contracts  being  measured  within  the  DoD  by  different  stakeholders,  (3)  how  should 
service  contracts  be  defined  and  measured  within  the  DoD.  We  conducted  41  interviews 
and  surveys  of  key  stakeholders.  Our  findings  reveal  no  standardized  definition  or 
measurements  for  success  of  service  contracts.  However,  some  salient  characterstics  of 
definitions  are  schedule,  maintain  costs,  and  well  defined  requirements.  With  respect  to 
measurements,  relevant  characteristics  included  performance  and  cost.  Furthermore,  we 
provide  recommendations  on  establishing  standardized  definitions  and  measurements  of 
success. 
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I.  INTRODUCTION 


A.  BACKGROUND 

Over  the  last  few  decades,  a  growing  trend  of  increased  dollar  value  in  service 
contracts  for  Department  of  Defense  (DoD)  contracting  has  become  apparent.  Relative  to 
supply  contracts,  services  acquisition  has  continued  to  grow  both  in  terms  of  dollar  value 
and  in  range  of  acquisitions.  This  trend  is  shown  in  Figure  1,  Department  of  Defense 
contracts  for  goods  and  services  (Rendon,  Apte,  &  Apte,  2012).  This  demonstrates  that 
between  fiscal  year  (FY)  2000  and  FY2010,  growth  in  service  contracts  more  than 
doubled.  Contract  obligations  rose  to  over  $387  billion  in  2008,  with  nearly  $200  billion 
spent  on  services  alone  (Hutton  &  Solis,  2009).  As  such,  a  management  and  oversight 
plan  and  clearly  defined  metrics  for  successful  service  contracts  are  important. 


FIGURE  1.  DOD’S  CONTRACTS  FOR  GOODS  AND  SERVICES 
(2000-2009) 
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Figure  1.  DoD  Contracts  for  Goods  and  Services 

(From  Rendon,  Apte,  &  Apte,  2012) 


Figure  2  clearly  demonstrates  this  growing  trend  in  service  contract  expenditures. 

Funding  spent  on  service  contracts  grew  steadily  from  1990-2010,  constituting  roughly 
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42%  of  the  total  spending  on  contracts  by  the  DoD,  exclusive  of  research  and 
development  services  contracts.  Notably,  service  contracts  showed  the  highest  growth  in 
percentage  of  expenditures  over  the  last  21  years,  with  a  rise  of  nearly  6.1%  annually 
(Ellman,  Livergood,  Morrow,  &  Sanders,  2011).  An  in-depth  study  of  services 
acquisition  will  help  develop  recommendations  to  evaluate  the  factors  of  success  in 
services  contracting. 
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Figure  2.  Growth  in  DoD  Service  Contract  Expenditures 

(From  Ellman  et  ah,  2011) 

The  definition  and  measurement  of  successful  service  contracts  should  align  with 
the  overarching  initiatives,  as  illustrated  by  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics  (USD[AT&Lj).  The  USD(AT&L)  outlined  these 
initiatives  in  his  June  2010  memorandum  on  acquisition  efficiency.  In  his  memorandum, 
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he  noted  that  the  Department  of  Defense  currently  spends  nearly  $400  billion  out  of  its 
$700  billion  budget  on  contracted  goods  and  services.  The  DoD  has  set  a  goal  to  achieve 
savings  of  more  than  $100  billion  over  the  next  five  fiscal  years.  The  USD(AT&L) 
describes  this  initiative  in  tenns  of  an  efficiency  and  cost-effectiveness  overhaul.  The 
DoD,  according  to  the  USD(AT&L),  has  the  potential  of  increasing  its  warfighter 
capability  by  2-3%  annually,  without  future  budget  increases.  In  addition,  identifying 
and  cutting  unproductive  or  low-value  programs  and  contracts  will  free  up  funding  to 
transfer  to  more  productive  warfighter  programs.  The  USD(AT&L)  states  that  the  United 
States  is  entering  a  period  of  budget  refonn,  stunting  the  budget  growth  of  the  previous 
decade.  The  ability  to  properly  manage  and  access  every  service  contract  is  essential  to 
reduce  inefficiencies  and  eliminate  nonperformance,  and,  consequentially,  to  achieve  the 
level  of  savings  required  by  the  DoD  savings  initiative  (USD[AT&L],  2010). 

B.  PURPOSE 

Our  purpose  with  this  research  was  to  conduct  a  comprehensive  investigation  into 
the  definition  and  metrics  of  a  successful  service  contract  from  the  perspective  of  various 
stakeholders.  Our  data  collection  methods  used  interviews  conducted  among 
stakeholders  within  the  DoD.  With  our  research  we  determine  if  the  stakeholders  define 
and  measure  the  success  of  service  contracts  differently.  The  results  of  this  project 
support  ongoing  research  being  conducted  by  the  Acquisition  Research  Program  at  the 
Naval  Postgraduate  School  concerning  the  DoD’s  management  of  service  contracts 
(Apte,  Apte  &  Rendon,  2010). 

C.  RESEARCH  QUESTIONS 

With  our  research  we  attempt  to  answer  the  following  questions  as  they  relate  to 
the  definition  and  measurement  of  successful  service  contracts  within  the  DoD,  and  as 
detennined  by  the  different  identified  stakeholders: 

1.  How  are  successful  service  contracts  within  the  DoD  being  defined  by 
different  stakeholders? 
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2.  How  are  service  contracts  being  measured  within  the  DoD  by  different 
stakeholders? 

3.  How  should  service  contracts  be  defined  and  measured  within  the  DoD? 

D.  BENEFITS  AND  LIMITATIONS  OF  THIS  RESEARCH 

In  addition  to  identifying  how  stakeholders  currently  define  and  measure  the 
success  of  service  contracts,  we  aim  to  provide  key  information  to  develop  sound 
protocol  and  metrics  for  future  service  contract  success.  By  determining  how  each 
stakeholder  within  the  DoD  defines  a  successful  service  contract,  we  endeavor  to 
distinguish  the  key  driving  factors  of  service  contract  success  that  lead  to  greater 
performance  and  savings. 

This  research  was  limited  by  the  sample  size  and  scope  of  the  population  of 
stakeholders  interviewed.  Of  the  numerous  organizations  within  the  DoD,  a  small 
percentage  of  these  stakeholders  participated  in  the  research. 

E.  RESEARCH  METHODOLOGY 

This  research  utilized  a  web-based  survey,  telephone  interviews,  and  personal 
interviews  of  the  various  stakeholders  within  DoD  service  contract  activities.  The  survey 
consisted  of  two  open-ended  questions  and  three  demographic  questions.  We  conducted 
a  review  of  literature  on  service  contract  management  and  stakeholder  theory.  We  then 
developed  a  survey  to  investigate  the  definition  and  measurement  of  successful  service 
contracts.  The  survey  was  deployed  across  three  contracting  commands  within  the  Navy. 
The  responses  of  all  participants  were  analyzed  and  examined  for  differences  and 
commonality.  We  then  developed  conclusions  and  provided  recommendations  for  the 
definition  and  measurement  of  a  successful  service  contract  process.  Finally,  we 
analyzed  and  examined  the  results  to  identify  and  categorize  how  stakeholders  define  and 
measure  successful  service  contracts. 
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F.  ORGANIZATION  OF  REPORT 

This  report  is  organized  into  five  chapters.  In  Chapter  I,  we  include  background 
information,  the  purpose  of  the  research,  our  research  questions,  the  benefits  and 
limitations  of  the  research,  and  the  research  methodology.  In  Chapter  II,  we  review  past 
and  current  literature  on  the  services  contracting  process.  We  describe  the  members  of 
the  acquisition  team,  their  roles  and  responsibilities,  as  well  as  their  goals  and  objectives. 
We  then  identify  the  deficiencies  in  service  contract  management  evidenced  through 
several  Government  Accountability  Office  (GAO)  and  Inspector  General  (IG)  findings. 
Further,  we  present  the  research  on  stakeholder  theory  and  how  it  relates  to  the  service 
contract  management  process.  In  Chapter  III,  we  outline  the  research  methodology, 
which  includes  our  data  collection  and  analytical  process.  In  Chapter  IV,  we  examine 
and  analyze  the  research  findings.  In  Chapter  V,  we  provide  the  summary,  conclusions, 
and  areas  for  further  research. 

G.  SUMMARY 

In  this  chapter,  we  provided  background  information  on  service  acquisition  within 
the  DoD,  the  purpose  of  the  report,  our  research  questions,  the  benefits  and  limitations  of 
our  research,  and  the  methodology  and  organization  of  the  report.  The  information  we 
provided  outlined  the  objectives  described  by  the  USD(AT&L)  and  how  these  objectives 
relate  to  the  definition  and  measurement  of  successful  service  contracts.  The  research 
questions  are  the  primary  focus  of  this  report.  In  Chapter  II,  we  review  past  and  current 
literature  on  the  service  contracting  process,  the  acquisition  team,  roles  and 
responsibilities  of  service  acquisition  personnel,  goals  and  objectives  of  service 
acquisition  personnel,  deficiencies  in  service  contracts,  and  stakeholder  theory. 
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II.  LITERATURE  REVIEW 


A.  INTRODUCTION 

In  Chapter  I,  we  established  the  foundation  of  this  research.  In  Chapter  II,  we 
introduce  past  and  current  literature  on  the  service  contracting  process,  the  acquisition 
team,  roles  and  responsibilities  of  service  acquisition  personnel,  goals  and  objectives  of 
service  acquisition  personnel,  deficiencies  in  service  contracts,  and  stakeholder  theory. 

B.  SERVICE  CONTRACTING  PROCESS 

Service  contract  management  is  defined  as  the  art  and  science  of  managing  an 
agreement  throughout  the  process  of  contracting  (Rendon  &  Snider,  2008).  The  contract 
management  process  can  be  described  utilizing  a  six-step  model.  Following  is  the  list  of 
phases  within  the  process  and  the  key  activities  of  each  phase: 

1.  Procurement  Planning:  Identification  of  which  organization  or  business 
needs  can  be  best  achieved  by  procurement  of  services  or  products 
external  to  the  organization.  Key  activities  include  determining  the  scope 
of  work,  completing  market  research,  technology  analysis,  and  funding 
determination,  and  creating  estimates  for  cost  and  schedule. 

2.  Solicitation  Planning:  Preparation  of  solicitation  documentation  to  support 
the  acquisition.  Key  activities  include  using  standardized  forms,  model 
contracts,  specifications  and  descriptions  of  items,  and  terms  and 
conditions  of  the  contract. 

3.  Solicitation:  Obtaining  bids  and  proposals  from  prospective  contractors  on 
how  to  meet  the  objectives  of  the  service  contract.  Pertinent  activities 
include  conducting  advertising  to  identify  new  sources  and  compiling  a 
list  of  interested  offerors. 

4.  Source  Selection:  Receipt  of  proposed  bids  and  application  of  selection 
criteria  for  supplier  products  or  services.  Activities  specific  to  this  phase 
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include  using  evaluation  criteria  centered  on  management,  technical,  and 
cost  factors  and  considering  an  offeror’s  past  performance  in  evaluating 
proposals. 

5.  Contract  Administration:  Ensuring  that  each  supplier’s  performance  is  in 
accordance  with  contractual  requirements.  Key  activities  include 
employing  an  integrated  team  approach  to  monitor  the  contractor’s  cost, 
schedule,  and  performance.  Additionally,  this  phase  includes  establishing 
a  process  for  administering  incentives  for  award  fee  provisions. 

6.  Contract  Closeout:  Validation  of  administrative  matters  pertaining  to 
completed  contracts.  Activities  unique  to  this  phase  includes  using 
checklists  and  forms  for  documentation  of  closed  contracts  and 
maintaining  lessons  learned  and  best  practices  for  use  in  future  contracts 
(Rendon  &  Garrett,  2005). 

The  outlined  activities  and  steps  in  the  service  contract  process  are  performed  by 
the  various  members  of  the  acquisition  team,  as  discussed  in  the  next  section. 

C.  THE  ACQUISITION  TEAM 

Throughout  the  federal  acquisition  service  contracting  process,  the  vision  is  to 
deliver  the  best  value  to  the  customer  in  a  timely  manner,  while  maintaining  the  public’s 
trust  and  fulfilling  public  policy  objectives  (FAR,  2012).  The  Federal  Acquisition 
Regulation  (FAR)  further  delineates  that  all  the  participants  in  the  acquisition  process 
should  work  together  as  a  team  and  should  be  empowered  to  make  decisions  within  their 
area  of  responsibility  (FAR,  2012).  The  FAR  defines  the  acquisition  team  as  consisting 
of  all  participants  in  government  acquisition,  including  not  only  representatives  of  the 
technical,  supply,  and  procurement  communities  but  also  the  customers  they  serve  and 
the  contractors  who  provide  the  products  and  services  (Federal  Acquisition  Regulation, 
2012).  According  to  the  FAR,  the  role  of  each  member  of  the  acquisition  team  is  to 
exercise  personal  initiative  and  sound  business  judgment  in  providing  the  best  value 
product  or  service  to  meet  the  customer’s  needs  (Federal  Acquisition  Regulation,  2012). 
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These  members  can  be  senior  agency  leaders,  government  personnel,  administrative 
employees,  and  even  support  employees  (U.S.  Merit  Systems  Protection  Board,  2005). 

Many  commercial  organizational  structures  have  incorporated  the  use  of  cross¬ 
functional  teams  in  an  effort  to  improve  communication,  coordination,  and  collaboration 
among  the  team  (Monczka,  Handheld,  Giunipero,  &  Patterson,  2011).  The  DoD 
implements  the  cross  functional  team  through  the  use  of  an  Integrated  Product  Team 
(IPT).  It  is  useful  to  note  that  the  IPT  is  primarily  used  in  contracting  for  products  to 
facilitate  the  process  of  meeting  cost,  perfonnance,  and  schedule  objectives  from  product 
concept  through  production,  including  field  support  (Press,  2012).  In  service  contracting 
the  IPT  is  a  team  composed  of  representatives  from  appropriate  functional  disciplines 
working  together  to  identify  and  resolve  issues;  make  sound,  timely  recommendations  in 
an  effort  to  facilitate  decision-making;  and  build  successful  programs  that  meet  the 
warfighter’s  needs  (Press,  2012). 

This  research  examines  the  internal  members  who  could  be  considered 
stakeholders  of  the  acquisition,  specifically  the  program  manager  (PM),  principal 
contracting  officer  (PCO),  and  contracting  officer  representative  (COR). 

1.  Project  Manager 

The  project  management  profession  is  principally  represented  by  the  Project 
Management  Institute  (PMI)  and  the  International  Project  Management  Association 
(IPMA);  each  group  has  its  own  representative  professional  certification  (Garrett,  2010). 
Project  management  is  defined  as  the  centralized,  coordinated  management  of  a  program 
to  achieve  the  program’s  strategic  objectives  and  benefits  (Project  Management  Institute, 
2008).  For  the  purposes  for  this  report,  the  project  manager  is  termed  program  manager 
(PM).  The  PM  has  the  ultimate  responsibility  for  all  cost,  schedule,  and  technical  aspects 
of  the  program  (Deneault  &  Stambaugh,  2000,  p.  22).  A  PM’s  responsibilities  cut  across 
multiple  acquisition  functional  areas  (e.g.,  business,  contracting,  facilities  engineering, 
information  technology,  life  cycle  logistics,  quality,  systems  planning)  as  well  as 
knowledge  in  other  technical  areas  (Krieger,  2011a,  p.  24).  In  most  companies,  PMs 
serve  as  multifunctional  team  leaders  on  one  or  more  projects,  responsible  for  achieving 
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the  desired  results  for  the  projects  (Garrett,  2010).  However,  most  PMs  lack  the  authority 
to  sign,  modify,  or  cancel  contracts  that  legally  bind  companies  (Garrett,  2010).  The 
same  is  true  for  DoD  PMs.  In  the  DoD,  the  PM  is  responsible  for  identifying  the 
requirements,  managing  the  project  processes,  planning,  monitoring,  and  executing  the 
project.  Further,  the  PM  is  accountable  for  addressing  the  various  needs,  concerns,  and 
expectations  of  the  stakeholders  as  the  project  is  planned  and  executed  (Project 
Management  Institute,  2008).  In  carrying  out  the  role  of  PM,  various  constraints  must  be 
balanced.  The  most  common  constraints  are  the  following:  scope,  quality,  schedule, 
budget,  resources,  and  risk  (Project  Management  Institute,  2008).  These  are  often 
competing  constraints  as  the  project  progresses.  Scope  is  defined  as  the  work  that  needs 
to  be  accomplished  to  deliver  the  service.  Included  in  scope  is  the  management  plan,  the 
description  of  how  the  project  scope  will  be  managed  and  controlled  as  well  as  a  baseline 
that  is  compared  to  the  actual  results.  Quality  is  the  results  toward  which  the  constraints 
are  driving.  Schedule  is  the  sequence  of  activities  and  duration  to  complete  the  service. 
Budget  is  managing,  estimating  and  controlling  costs  of  the  service.  Resources  refer  to 
the  human  personnel  constraint  that  must  be  managed.  In  all  projects,  risk  must  be 
identified,  monitored,  and  managed.  All  of  these  common  constraints  and  their 
interdependencies  are  best  illustrated  in  the  project  management  triangle  shown  in  Figure 
3.  Each  side  represents  a  constraint.  These  three  constraints  are  often  competing 
constraints:  increased  scope  typically  means  increased  time  and  increased  cost,  a  tight 
time  constraint  could  mean  increased  costs  and  reduced  scope,  and  a  tight  budget  could 
mean  increased  time  and  reduced  scope  (Sekhar,  2010).  The  DoD  5000  Series  is  a 
regulatory  document  that  provides  guidance  and  policy  for  the  management  of  defense 
acquisitions.  Program  management  is  the  management  of  all  of  the  project  goals  and 
objectives,  while  honoring  the  preconceived  constraints  as  well  as  ensuring  compliance 
with  the  5000  Series  directives  in  the  defense  acquisition  guidebook  (Krieger,  2011a). 
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Figure  3.  Project  Management  Triangle 

(From  Font,  V.,  2010  ) 

According  to  A  Guide  to  Best  Practices  for  Contract  Administration,  program 
management  activities  include  planning,  organizing,  securing,  and  managing  resources  to 
achieve  specific  goals  (Office  of  Federal  Procurement  Policy,  1994).  The  PM  adapts  to 
various  internal  procedures  of  the  contracting  process  and  forms  close  links  with  the 
various  stakeholders.  To  achieve  client  satisfaction,  it  is  essential  that  the  PM  realize  key 
issues  of  cost,  schedule,  and  performance.  In  government  acquisition,  the  PM  should 
obtain  integrated  cost  and  schedule  performance  data  at  an  appropriate  level  in  order  to 
monitor  program  execution.  The  PM  should  require  contractors  and  government 
activities  to  use  internal  management  control  systems  that  accomplish  the  following: 

•  relate  time-phased  budgets  to  specific  tasks  identified  in  the  statement  of 
work; 

•  produce  data  that  indicate  work  progress; 

•  properly  relate  cost,  schedule,  and  technical  accomplishment;  and 

•  produce  data  that  are  valid,  timely,  and  auditable.  (USD[AT&L]),  2008) 

In  government  acquisition  the  PM  is  ultimately  accountable  to  the  end  user  and 

Congress.  Therefore,  the  PM  activities  include  extensive  coordination  with  internal  and 

external  stakeholders  (Deneault  &  Stambaugh,  2000,  p.  23).  The  PM  owns  the 

acquisition  strategy  which  drives  the  acquisition  plan  (Krieger,  2011a,  p.  42).  This 

strategy  shall  be  in  writing  and  prepared  in  accordance  with  all  the  requirements  of 

subpart  7.1.  of  the  FAR  (2012).  The  strategy  is  the  PM’s  overall  plan  for  satisfying  the 
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mission  need  in  the  most  effective,  economical,  and  timely  manner,  so  before  there  is  a 
contract,  the  PM  owns  the  acquisition  strategy  that  the  contracting  officer  will  seek  to 
implement  (Krieger,  2011a,  p.  42). 

In  government  acquisition,  the  PM  is  seen  as  successful  if  they  continue  to 
perpetuate  their  program  through  the  acquisition  life  cycle.  This  focus  tends  to  take 
precedence  over  reporting  realistic  program  status.  PMs  are,  by  nature,  becoming 
program  advocates.  A  PMs’  future  assignments  and  promotions  depend  on  the  success  of 
the  program,  and  it  becomes  very  difficult  for  the  PM  to  “blow  the  whistle,”  hoping  that 
they  will  be  transferred  before  the  true  costs  of  the  project  become  known  (Fox,  Hirsch, 
Krikorian,  &  Schumacher,  1994).  The  PM  receives  guidance  and  oversight  from  the  all 
three  branches  of  the  government;  serious  problems  with  a  program  can  significantly 
jeopardize  congressional  and  Office  of  the  Secretary  of  Defense  support  for  the  program 
(Krieger,  2011a).  PMs  sometimes  lack  realism  and  have  undue  optimism;  a  program 
cancellation  or  reduction  in  scope  is  perceived  as  a  PM’s  failure  by  their  service 
superiors.  Therefore,  a  conflict  exists  between  reporting  realistic  program  status  and 
doing  what  is  necessary  to  keep  programs  funded  and  moving  through  the  acquisition  life 
cycle  (Fox  et  ah,  1994). 

2.  Principal  Contracting  Officer 

The  contract  management  profession  is  represented  by  three  professional 
associations:  the  National  Contract  Management  Association  (NCMA),  the  Institute  for 
Supply  Management  (ISM),  and  the  International  Association  of  Contract  and 
Commercial  Management  (IACCM).  Each  group  has  its  own  professional  certifications 
(Garrett,  2010).  Contract  management  is  a  career  field  that  takes  significant  on-the-job 
training  to  become  truly  competent  (Deneault  &  Stambaugh,  2000,  p.  21).  Only  a  few 
educational  institutions  offer  degrees  or  professional  certificates  in  government  contract 
management,  and  education  does  not  always  equate  to  experience  (Deneault  & 
Stambaugh,  2000,  p.  21)  A  contract  manager  must  be  able  to  integrate  functional  inputs 
into  a  solicitation  and  the  resulting  contract  (Krieger,  2011b,  p.  25).  Despite  the  fact 
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contract  managers  are  seldom  responsible  for  daily  project  planning  or  operation,  they  are 
authorized  to  enter  their  organizations  into  legally  binding  contractual  arrangements 
(Garrett,  2010).  Contract  managers  must  be  able  to  examine  the  contractual  meaning  of 
pre-contractual  events  and  documents.  Contract  managers  must  discern  the  objectives, 
needs,  limitations,  and  even  prejudices  of  other  stakeholders  (Hirsch,  1986).  When  a 
contract  manager  has  this  sensitivity,  the  contract  manager  can  reduce  strife  and  gain 
empathy  towards  other  members  in  the  acquisition  team  (Hirsch,  1986).  A  contract 
manager  must  have  the  skills  to  answer  the  following  questions:  (1)  What  is  the  work? 
(2)  What  are  and  where  do  we  find  the  skills  to  perform  the  tasks?  (3)  How  do  we  ensure 
the  work  is  done  effectively  at  the  lowest  price  (Cohen  &  Eimicke,  2008)?  Most 
organizations  empower  one  or  more  employees  in  the  role  of  contract  management,  and 
in  some  agencies  a  relatively  small  number  of  high-level  officials  are  designated 
contracting  managers  solely  by  virtue  of  their  positions  (National  Archives  and  Records 
Administration,  2012).  This  employee  (the  contract  manager)  is  referred  to  in  contract 
law  as  the  agent;  the  source  of  authority  is  referred  to  as  the  principal  (Garrett,  2010).  In 
DoD  contracting  this  agent  is  called  the  principal  contracting  officer  (PCO)  and  the 
principal  is  the  federal  government.  PCOs  have  the  sole  authority  of  the  government  to 
legally  bind,  enter  into,  administer,  and  tenninate  contracts  and  make  related 
determinations  and  findings.  PCOs  may  bind  the  government  only  to  the  extent  of  the 
authority  delegated  to  them  in  clear,  written  instructions  as  to  the  limits  of  their  authority 
through  their  warrant  (Federal  Acquisition  Regulation,  2012).  The  PCO  is  responsible 
for  meeting  the  conditions  of  the  contract,  and  therefore  spends  more  time  on  business 
and  legal  issues  and  contract  administration  (Deneault  &  Stambaugh,  2000,  p.  23).  A 
PCO’s  warrant  is  taken  very  seriously;  it  distinguishes  an  individual  for  taking  on  a 
significant  amount  of  responsibility  and  accountability. 

Warrants  are  not  taken  for  granted  after  they  are  received  because  they  can 
be  lost  through  job  changes  or  the  belief  by  superiors  that  a  person  lacks 
the  skills  or  knowledge  necessary  to  uphold  the  position.  When  a  warrant 
is  lost  a  contracting  officer  has  no  choice  but  to  change  careers  or  earn 
another  warrant.  (Deneault  &  Stambaugh,  2000,  p.  22) 
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The  government’s  warrant  authorizes  the  PCO  as  the  only  individual  who  can  financially 
obligate  the  organization  and  requires  them  to  be  involved  in  all  communications 
(Deneault  &  Stambaugh,  2000,  p.  22).  Thus,  a  significant  amount  of  training  is  required 
on  the  constraints  of  the  law  and  requirements  of  the  contract  (Deneault  &  Stambaugh, 
2000,  p.  22).  No  contract  shall  be  entered  into  unless  the  PCO  ensures  that  all 
requirements  of  law,  executive  orders,  regulations,  and  all  other  applicable  procedures, 
including  clearances  and  approvals,  have  been  met  (Federal  Acquisition  Regulation, 
2012).  Further,  the  PCO  is  responsible  for  ensuring  performance  of  all  necessary  actions 
for  effective  contracting,  to  include  ensuring  compliance  with  the  terms  of  the  contract 
and  safeguarding  the  interests  of  the  United  States  in  its  contractual  relationships  while 
ensuring  adherence  to  procurement  laws  and  regulations,  given  their  ability  and 
instructed  by  Title  48  of  the  Code  of  Federal  Regulations  (CFR),  the  FAR  (2012),  and  the 
Defense  Federal  Acquisition  Regulations  Supplement  (DFARS;  2010).  Additionally,  the 
PCO  is  responsible  for  assisting  in  clarifying  agency  needs,  market  research,  contract 
methods,  acquisition  planning,  competing  and  meeting  source  selection  requirements,  and 
conducting  negotiations  (Hirsch,  1986).  In  order  to  perfonn  these  responsibilities, 
contracting  officers  should  be  allowed  wide  latitude  to  exercise  business  judgment.  The 
FAR  specifically  mandates  that  contracting  officers  shall 

(a)  Ensure  that  the  requirements  of  1.602- 1(b)  (FAR  2012)  have  been  met, 
and  that  sufficient  funds  are  available  for  obligation; 

(b)  Ensure  that  contractors  receive  impartial,  fair,  and  equitable  treatment;  and 

(c)  Request  and  consider  the  advice  of  specialists  in  audit,  law,  engineering, 
information  security,  transportation,  and  other  fields,  as  appropriate. 

(d)  Unless  the  contracting  officer  retains  and  executes  the  COR  duties,  in 
accordance  with  agency  procedures,  designate  and  authorize,  in  writing,  a 
COR  on  all  contracts  and  orders  other  than  those  that  are  firm-fixed  price, 
and  for  firm-fixed-price  contracts  and  orders  as  appropriate  (Federal 
Acquisition  Regulation,  2012). 
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3.  Project  Manager  and  Principal  Contracting  Officer  Similarities  and 
Differences 

Often  PMs  and  PCOs  work  in  a  matrix  organization  characterized  by 
multifunctional  teams.  PMs’  and  PCOs’  roles  often  overlap  in  terms  of  competencies  and 
responsibilities.  This  is  evident  in  looking  at  their  professional  certification  programs, 
bodies  of  knowledge,  and  day-to-day  interactions  (Garrett,  2010).  On  the  acquisition 
team,  the  PM  is  responsible  for  what  needs  to  be  done  to  execute  the  program  through  the 
phases  outlined  in  the  acquisition  strategy;  the  PCO  and  the  other  members  of  the 
acquisition  team  implement  the  strategy  through  the  contract  (Krieger,  2011b,  p.  42).  In 
successful  programs  PMs  and  PCOs  work  together  effectively,  yet  there  is  a  great  deal  of 
conflict.  In  some  instances,  the  PM  is  separated  from  the  supported  team  members 
(Deneault  &  Stambaugh,  2000,  p.  22).  PMs  rely  on  their  functional  departmental  support 
to  achieve  program  success.  In  this  interaction,  there  is  a  potential  for  conflicts  between 
the  stakeholders’  interests;  the  PM  is  overall  in  charge,  but  the  PCO  is  the  only  member 
that  can  challenge  the  PMs  authority  (Deneault  &  Stambaugh,  2000,  p.  22).  PMs  often 
feel  that  their  flexibility  is  constrained  by  the  conservative  interpretations  of  the  PCO 
(Deneault  &  Stambaugh,  2000).  PM  and  PCO  roles  overlap  at  the  intersection  where 
program  execution  meets  contracting  officers’  authority.  Flexibility  and  adaptability  have 
become  prerequisite  traits  for  PMs  and  PCOs  to  succeed  (Deneault  &  Stambaugh,  2000, 

p.  22). 

As  previously  discussed,  the  PM  is  focused  on  meeting  cost,  schedule,  and 
performance  objectives  while  progressing  through  the  project  life  cycle.  The  PCO  is 
responsible  for  ensuring  all  aspects  of  the  law  are  complied  with  and  protecting  the 
government’s  interests.  These  two  different  perspectives  are  often  in  conflict  with  each 
other.  The  PCO’s  objectives  do  not  align  with  the  PM’s.  This  could  be  the  reason  why 
one  could  say  the  PCO  is  the  unpopular  player  on  the  acquisition  team  (Hirsch,  1986). 

4.  Contracting  Officer  Representative 

PCOs  designate  and  authorize  the  CORs.  As  the  technical  subject  matter  expert 
on  the  acquisition  team,  the  COR  is  an  integral  stakeholder  in  the  contracting  process  and 
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is  the  first  line  of  surveillance  on  government  contracts.  The  COR  also  plays  a  critical 
role  in  contract  administration.  Agencies  and  departments  have  many  different  titles  to 
describe  the  COR.  Other  titles  used  for  this  role  are  government  technical  representative 
(GTR)  and  government  technical  evaluator  (GTE;  Office  of  Federal  Procurement  Policy, 
1994).  For  purposes  of  this  thesis,  we  use  COR  because  it  is  the  most  common  title  for 
this  function.  The  COR  is  nominated  by  the  PCO,  as  early  as  practicable  in  the 
acquisition  process  (FAR,  2012).  The  COR’s  administrative  duties  range  from  simple  to 
complex,  dependent  on  the  type  of  contract,  contractor  performance,  and  the  nature  of  the 
work.  The  COR  functions  as  the  “eyes  and  ears”  of  the  contracting  officer  and  monitors 
technical  perfonnance,  reporting  any  potential  or  actual  problems  to  the  contracting 
officer  (Office  of  Federal  Procurement  Policy,  1994).  A  COR  must  stay  in  close 
communication  with  the  contracting  officer,  relaying  any  information  that  may  affect 
contractual  commitments  and  requirements.  The  FAR  (2012)  specifically  mandates  that 
a  COR  must  meet  the  following  criteria: 

•  shall  be  a  government  employee,  unless  otherwise  authorized  in  agency 
regulations; 

•  shall  be  certified  and  maintain  certification; 

•  must  be  qualified; 

•  may  not  re-delegate  responsibility  to  perfonn  functions  that  have  been 

delegated; 

•  has  no  authority  to  make  any  commitments  or  changes  that  affect  price, 
quality,  quantity,  delivery,  or  other  terms  and  conditions  of  the  contract; 

•  nominated  by  the  requirements  official. 

It  should  be  noted  that  although  the  COR  is  a  critical  member  of  the  acquisition 
team  as  defined  by  the  FAR,  the  COR  is  not  a  member  of  the  acquisition  work  force,  as 
defined  by  the  Defense  Acquisition  Work  Force  Improvement  Act  (DAWIA;  1990). 

The  COR  provides  the  technical  expertise  necessary  for  successful  contracting 
and  plays  a  critical  role  in  affecting  the  outcome  of  the  contract  administration  process,  as 
well  as  ensuring  maximum  return  on  contract  dollars.  The  following  is  an  example  taken 

from  the  researchers  own  experience  of  COR  duties  for  a  consulting  services  contract: 
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•  Control  all  government  technical  interfaces  with  the  contractor. 

•  Ensure  that  a  copy  of  all  government  technical  correspondence  is 
forwarded  to  the  contracting  officer  for  placement  in  the  contract 
(delivery/task  order)  file. 

•  Promptly  furnish  documentation  on  any  requests  for  change,  deviation,  or 
waiver,  whether  generated  by  the  government  or  the  contracting  officer 
(and  ordering  officer)  for  their  action. 

•  Determine  causes  when  the  contract  is  not  progressing  as  expected  and 
make  recommendations  to  the  contracting  officer  for  corrective  action. 

•  Monitor  contractor  performance  to  ensure  individual  contractor  employees 
are  of  the  skill  levels  required  and  are  actually  perfonning  at  the  levels 
charged  against  the  contract  during  the  performance  period. 

•  Monitor  contractor  performance  to  ensure  that  the  labor  hours  charged 
against  the  contract  are  consistent  and  reasonable  for  the  effort  completed 
and  that  any  travel  charged  was  necessary  and  actually  occurred. 

•  Monitor  government  furnished  property.  Ensure  that  property  provided  to 
the  contractor  is  authorized  by  the  contract. 

•  Complete  the  COR  Report  of  Contractor’s  Performance  in  accordance 
with  the  schedule  established  in  the  contract  administration  plan  for  a 
contract. 

The  PM,  PCO,  and  COR  are  critical  stakeholders  responsible  for  overall  program 
success.  These  stakeholders  hold  key  decision-making  positions  and  at  times  may 
overexert  their  influence  based  on  their  positions,  resulting  in  difficult  and  contentious 
conflicts  (Deneault,  &  Stambaugh,  2000). 

As  previously  discussed,  each  stakeholder  has  conflicting  goals  and  objectives,  as 
well  as  different  guiding  policies  and  directives.  These  all  could  lead  to  deficiencies  in 
DoD  services  contracting.  We  will  present  these  deficiencies  in  the  next  section. 
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D.  DEFICIENCIES  IN  SERVICE  CONTRACTS 


The  DoD’s  contract  obligations  have  doubled  between  fiscal  years  2001  and  2008 
to  over  $387  billion  with  $200  billion  expended  on  services  (Government  Accountability 
Office  [GAO],  2009).  Commensurately,  the  acquisition  workforce  declined  from 
500,000  to  200,000  personnel  in  2006  (Gansler,  2011,  p.  237).  The  downsizing  of  the 
defense  acquisition  workforce  has  reduced  a  qualified  contracting  and  acquisition 
workforce  necessary  to  manage  the  increased  service  contract  workload  (GAO,  2002b, 
2009a).  Human  capital  problems  are  debilitating  many  agencies  and  threaten  the  ability 
of  others  to  perform  their  missions  efficiently  and  effectively  (GAO,  2001).  Both  the 
GAO  and  the  DoD  Inspector  General  (IG)  have  indicated  that  failing  to  maintain  an 
adequate  workforce  to  manage  the  billion-dollar  acquisitions  increases  the  risk  of  poor 
acquisition  outcomes  and  the  likelihood  of  fraud,  waste,  and  abuse  (GAO,  2009.  From 
2001  to  2009  the  GAO  has  issued  16  reports  identifying  deficiencies,  trends,  and 
challenges  in  contract  management.  In  addition,  the  DoD  IG  issued  142  reports  on 
deficiencies  in  the  DoD  acquisition  and  contract  administration  process.  Deficiencies 
include  selection  of  an  inappropriate  contract  type  allocating  unnecessary  risk  to  the 
government.  DoD  contracting  officers  have  selected  risk-laden  cost  contracts  for  services 
in  which  a  fixed  contract  type  could  have  been  used,  diverting  the  majority  of  the  risk  to 
the  contractor  (GAO,  2001;  DoD  IG,  2009).  Additionally,  the  GAO  and  DoD  IG  reports 
have  expanded  on  the  government’s  lack  of  adequate  market  research  relevant  to 
determining  the  proper  contracting  strategy  during  the  procurement  planning  of  service 
contracts  (GAO,  2002a;  DoD  IG,  2009). 

Requirements  management  is  required  to  effectively  define  and  meet  the 
customers’  needs  and  expectations.  Despite  the  importance  of  requirements  management, 
the  DoD  IG  and  GAO  have  indicated  poorly  defined  requirements  and  inadequate 
requirements  management  as  a  problem  in  service  contracting  (GAO,  2007b;  DoD  IG 
2009).  Although  the  implementation  of  project  management  tools  and  processes  such  as 
cross  functional  teams  is  considered  a  best  practice  in  service  contracts  and  would 
improve  the  coordination  and  management  of  service  acquisitions,  the  GAO  has  indicated 
the  DoD  lacks  management  structure  and  processes  for  managing  service  contracts 
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(GAO,  2002b,  2007b;  DoD  IG,  2009).  The  GAO  has  described  the  DoD’s  current 
approach  to  service  contract  management  as  reactive  and  not  fully  addressing  the  key 
factors  of  success  (GAO,  2007b).  Oversight  and  surveillance  are  prudent  to  achieving 
contractors’  adequate  performance  of  services  and  assist  in  precluding  any  contractor 
performance  problems.  Lack  of  oversight  compromises  the  government’s  ability  to 
provide  complete  value  to  its  constituents.  Yet,  DoD  IG  and  GAO  reports  have 
consistently  identified  issues  in  service  contract  administration  and  oversight  (GAO, 
2005,  2007a,  2007b;  DoD  IG,  2009).  According  to  the  GAO,  the  poor  management  of 
service  contracts  has  undermined  the  government’s  ability  to  obtain  a  good  value  for  the 
money  spent  and  contributed  to  the  GAO’s  decision  to  designate  contract  management  a 
high-risk  area  for  the  DoD  (GAO,  2001,  2007c,  2011). 

The  DoD  IG’s  and  GAO’s  finding  on  deficiencies  in  the  service  contract  process 
can  be  further  explained  by  utilizing  stakeholder  theory. 

E.  STAKEHOLDER  THEORY 

1.  Definition  and  Implication  for  DoD  Services  Contracting 

Stakeholder  theory  can  be  used  to  analyze  and  discuss  the  similarities  and 
differences  that  we  anticipate  will  exist  in  defining  and  measuring  the  success  of  a  service 
contract  between  the  PM,  PCO,  and  COR.  In  this  section,  we  describe  the  foundations  of 
stakeholder  theory  and  how  that  theory  relates  to  DoD  service  contracting.  Using 
stakeholder  theory  can  lead  to  a  better  understanding  of  both  the  conflicting  and  common 
interests  the  PM,  PCO,  and  COR  might  have.  This  can  help  acquisition  professionals  to 
better  understand  why  various  stakeholders  in  DoD  service  contracting  define  and 
measure  the  success  of  service  contracts  differently. 

There  have  been  numerous  definitions  of  what  a  stakeholder  is,  but  no  definition 
is  more  important  than  that  of  Edward  Freeman  (1984).  Considered  a  pioneer  in  the  field 
of  stakeholder  theory,  Freeman  defined  a  stakeholder  as  any  group  or  individual 
person(s)  that  can  be  affected  by  an  organization  or  identity  achieving  its  mission  or 
objectives  (Freeman,  1984).  Freeman,  Wicks,  and  Parmar  (2004)  of  the  Darden  School 
of  Business,  have  stated  that  stakeholder  theory  illustrates  clear  conflicts  of  interests 
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between  the  different  stakeholders  within  a  project.  They  stated  that  these  conflicts  of 
interests  can  have  negative  effects  (such  as  inefficiencies  and  waste)  on  numerous 
projects.  Differing  goals  and  objectives  of  the  various  stakeholders  drive  these  negative 
impacts  (Freeman  et  ah,  2004).  Donaldson  and  Preston  (1995)  further  explain  that 
organizations  which  actively  manage  often  conflicting  interests  of  internal  stakeholders 
fare  far  better  in  traditional  measures  of  success,  such  as  return  on  investment  and  profits, 
than  those  who  do  not  (Donaldson  &  Preston,  1995).  What  this  means  for  DoD  services 
contracting  is  quite  apparent.  With  the  differing  goals,  objectives,  and  responsibilities  of 
the  various  stakeholders  (PM,  PCO,  COR)  within  DoD  services  contracting,  it  is  prudent 
to  assume  that  conflicts  will  arise  with  regard  to  how  a  service  contract  is  processed,  as 
well  as  answering  the  questions  of  how  a  successful  service  contract  is  defined  and 
measured.  While  profit  and  return  on  investment  are  not  applicable  to  defense  services 
contracting,  it  is  important  to  note  the  impact  that  managing  stakeholder  interests  will 
have  on  the  public  sector,  specifically  in  the  area  of  public  interest  by  reducing  integrity, 
accountability,  and  transparency  in  the  contracting  process.  By  managing  the 
stakeholders’  conflicting  interests,  the  DoD  may  achieve  some  of  the  additional 
efficiencies  received  by  its  counterparts  in  the  private  sector. 

2.  Stakeholder  Management 

Stakeholder  management  is  described  as  the  management  of  the  individuals  and 
institutions  that  share  a  stake  or  interest  in  a  project  (Cleland,  1986).  In  the  case  of  DoD 
services  contracting,  the  “project”  is  the  acquisition  of  the  service.  Cleland  describes  the 
principle  justification  for  utilizing  a  stakeholder  management  perspective  to  be  the  need 
to  recognize  the  ability  of  key  stakeholders  in  influencing  projects  (Cleland,  1986).  As 
presented  earlier  in  this  chapter,  the  PM,  PCO,  and  COR  all  have  different  roles  and 
responsibilities  within  DoD  services  contracting.  As  such,  they  routinely  have  conflicting 
interests  and  objectives  of  a  service  contract,  which  in  turn  may  lead  to  conflicting 
definitions  and  measures  of  a  successful  service  contract.  Stakeholder  theory  holds  that 
the  various  stakeholders  will  routinely  have  not  just  differing  objectives  and  motives  for 
outcomes  within  a  project,  but  additionally  will  have  differing  ideas  as  to  which  factors 

are  the  most  important  for  determining  success  of  a  project.  This  can  present  additional 
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future  challenges  when  balancing  the  tradeoffs  between  the  perfonnances  criteria  (such 
as  cost  and  schedule)  of  a  contract  (Project  Management  Institute,  2008).  Understanding 
stakeholder  theory  and  how  these  different  stakeholders  maintain  conflicting  interests  that 
will  impact  the  contract  allows  us  to  better  prepare  for  mitigating  the  adverse  results  of 
such  conflicting  objectives.  This  relates  directly  to  our  research  into  the  definitions  and 
metrics  for  a  successful  service  contract,  as  held  by  its  chief  stakeholders:  PMs,  PCOs, 
and  CORs.  Cleland  states  that  positive  stakeholder  management  can  lead  to  cooperation 
within  the  project  (or  service  contract)  between  the  different  stakeholders,  resulting  in 
enhanced  project  objective  achievement.  Lack  of  positive  stakeholder  management  will 
result  in  the  reduction  in  project  objective  achievement  (Cleland,  1986).  Translated  into 
DoD  services  contracting,  this  means  that  proper  managing  and  mitigating  of  conflicting 
stakeholder  objectives  will  lead  to  more  effective  and  efficient  services  contracts. 
Additionally,  this  may  lead  to  a  more  standardized  definition  and  measure  of  successful 
service  contracts  by  DoD  PMs,  PCOs  and  CORs.  Understanding  this  concept  allows 
acquisition  professionals  to  predict  and  even  aid  in  control  of  the  conflicting  goals  of 
different  stakeholders  within  DoD  services  contracting. 

One  approach  to  standardized  measures  would  be  S.M.A.R.T.  metrics. 
S.M.A.R.T.  is  a  tool  utilized  by  corporate  officers,  managers,  and  supervisors  in  helping 
to  determine  quantifiable  metrics  and  objectives  for  their  project  and  company  mission. 
The  establishment  of  objectives  and  the  development  of  action  plans  are  extremely 
important  steps  in  any  organizations  management  process  (Doran,  1981).  Each  metric 
should  have  the  following  attributes: 

•  Specific:  It  targets  a  specific  area  for  improvement 

•  Measurable:  It  quantifies  or  provides  an  indicator  of  progress. 

•  Assignable:  Someone  is  determined  to  have  responsibility  for  it. 

•  Realistic:  It  can  realistically  be  obtained,  given  resource  constraints. 

•  Time  related:  It  is  specific  to  when  the  results  can  be  achieved.  (Doran, 
1981) 
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F.  SUMMARY 

In  this  chapter,  we  introduced  past  and  current  literature  on  the  service  contracting 
process,  the  acquisition  team,  roles  and  responsibilities  of  service  acquisition  personnel, 
goals  and  objectives  of  service  acquisition  personnel,  deficiencies  in  service  contracts, 
and  stakeholder  theory. 

The  next  chapter  describes  our  research  methodology,  including  the  type  of 
analysis,  the  location  of  data  collection,  and  the  interview  questions. 
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III.  RESEARCH  METHODOLOGY 


A.  INTRODUCTION 

In  this  chapter,  we  provide  an  overview  of  how  we  collected  and  analyzed  our 
data  in  order  to  achieve  our  objectives  and  answer  the  research  questions  discussed  in 
Chapter  I.  We  discuss  the  formulation  of  the  interview  questions,  the  survey,  and  our 
analytical  process.  In  this  chapter,  we  also  include  a  description  of  the  qualitative 
methods  we  used  in  analyzing  the  data  collected  from  the  Navy  contracting  activities. 
Our  purpose  was  to  conduct  an  exploratory  research  analysis  of  the  definition  and 
measurement  of  successful  services  contracts.  The  objective  of  the  research  was  to  build 
upon  the  understanding  developed  in  prior  research  and  explain  what  metrics  are  utilized 
to  help  identify  the  factors  that  influence  the  efficiency  and  effectiveness  of  service 
contracts.  We  analyzed  the  collected  data  qualitatively  in  order  to  draw  conclusions 
about  the  definition  and  measurement  of  success  of  service  contracts,  as  well  as  the 
commonalities  and  distinctions  among  the  stakeholders. 

B.  DATA  COLLECTION 

We  conducted  interviews  and  surveys  with  the  PMs,  PCOs,  and  CORs  located  at 
and  associated  with  the  following  contracting  commands:  Naval  Sea  Systems  Command 
(NAVSEA),  Naval  Air  Systems  Command  (NAVAIR),  and  Naval  Fleet  Logistics  Center 
(FLC)  Philadelphia.  Table  1  identifies  the  number  of  stakeholders  interviewed  and 
surveyed. 
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Table  1.  Interviews  and  Surveys  Conducted 


STAKEHOLDER 

# 

INTERVIEWED 

# 

SURVEYED 

TOTAL 

RESPONDENTS 

CONTRACTOR 

2 

N/A 

2 

PRINCIPAL  CONTRACTING  OFFICER/ 
CONTRACT  SPECIALISTS 

22 

N/A 

22 

CONTRACTING  OFFICER  REPRESENTATIVE 

13 

1 

14 

PROGRAM  MANAGER 

1 

2 

3 

1.  Participating  Commands 

NAVSEA’s  overarching  mission  is  to  engineer,  build,  buy,  and  maintain  ships, 
submarines,  and  combat  systems  to  meet  the  United  States  Navy’s  current  and  future 
operational  needs.  NAVSEA’s  fiscal  budget  compromises  25%  of  the  Navy’s  entire 
budget,  at  nearly  $30  billion.  NAVSEA  manages  over  150  acquisition  programs,  to 
include  foreign  and  domestic  military  sales.  NAVSEA  is  an  essential  system  command 
for  the  United  States  Navy,  providing  efficient  resources  and  support  for  the  nation 
(NAVSEA,  2012). 

NAVAIR’s  mission  is  to  provide  full  life  cycle  support  of  naval  aviation  aircraft, 
weapons,  and  systems  for  the  United  States  Navy;  to  provide  the  right  capability,  at  the 
right  time,  at  the  right  cost.  This  support  includes  research,  design,  development,  systems 
engineering,  acquisition,  test  and  evaluation,  training,  repair,  and  logistics  support.  They 
further  provide  support  to  their  program  executive  officers  with  their  assigned  duties  of 
meeting  cost,  schedule,  and  performance  requirements  within  their  respective  programs 
(NAVAIR,  2012). 

FLC  Philadelphia  is  a  subordinate  command  of  Naval  Supply  Systems  Command. 
FLC  Philadelphia’s  mission  is  to  deliver  sustained  global  logistics  resources  to  the  United 
States  Navy  and  the  joint  warfighter.  FLC  Philadelphia  manages  supply  chains  that 
provide  material  for  Navy  aircraft,  surface  ships,  submarines,  and  their  weapon  systems. 
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Additionally,  they  provide  logistics  support  services,  material  management  and 
warehousing  services,  contracting  and  acquisition,  as  well  as  all  food  service  operational 
support  ashore  and  afloat  (Naval  Supply  Systems  Command,  2012). 

2.  Interview  Questions 

We  developed  interview  questions  and  a  survey  to  answer  the  following  core 
research  questions: 

•  What  is  the  definition  of  a  successful  service  contract? 

•  How  is  the  success  of  a  service  contract  being  measured? 

The  survey  and  interview  questions  consisted  of  the  following  five  questions: 

1 .  What  is  your  branch  of  service  or  service  affiliation? 

2.  What  is  your  current  functional  role? 

3.  What  is  your  DAWIA  level  certification? 

4.  How  do  you  define  a  successful  service  contract  and  what  factors  are 
included  in  your  definition? 

5.  How  do  you  measure  the  success  of  a  service  contract  and  what  metrics 
are  included  in  your  measurements? 

The  first  two  questions  of  the  survey  and  interviews  identify  the  demographics  of 
the  respondents  to  establish  broad  categories.  The  third  question,  on  DAWIA  level 
certification,  gives  insight  into  the  respondents’  level  of  training,  experience,  and 
education.  The  DAWIA  establishes  a  procedure  through  which  acquisition  workforce 
personnel  are  recognized  as  having  achieved  qualification  in  their  core  discipline. 
Certification  is  the  procedure  through  which  a  DoD  component  determines  that  an 
employee  has  met  the  education,  training,  and  experience  standards.  These  standards  are 
required  for  a  career  in  govermnent  acquisition,  technology,  and  logistics  fields.  DAWIA 
level  certifications  are  categorized  as  Level  I,  Level  II,  and  Level  III.  The  remaining  two 
questions  address  the  primary  purpose  of  our  research,  to  detennine  how  stakeholders 
define  and  measure  the  success  of  service  contracts. 
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C.  ANALYTICAL  PROCESS 

We  chose  open-ended  questions  to  gain  qualitative  data  for  our  research  and  to 
accommodate  differentiated  responses  from  varied  stakeholders.  In  Chapter  IV,  we 
present  this  qualitative  data  in  graphical  and  tabular  fonnats.  By  consolidating  the  data 
into  categories,  we  were  able  to  conduct  further  analysis.  We  identify  commonalities  and 
relationships  in  the  data  to  determine  the  answers  to  our  research  questions,  identified  in 
Chapter  I.  Additionally,  we  determine  if  the  stakeholders  (PM,  PCO,  and  COR)  are  in 
fact  utilizing  effective  and  quantifiable  S.M.A.R.T. 

D.  SUMMARY 

In  Chapter  III,  we  identified  which  Navy  organizations  were  surveyed  and 
interviewed,  the  formulation  of  our  survey  and  interview  questions,  and  how  the  data  was 
collected  and  analyzed.  In  Chapter  IV,  we  present  the  data,  analyze  of  the  findings,  and 
make  recommendations. 
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IV.  INTERVIEW  INSTRUMENT,  RESULTS  AND  ANALYSIS 


A.  INTRODUCTION 

In  this  chapter,  we  examine  the  responses  to  the  questions  answered  in  our 
interviews.  The  objective  of  this  research  was  the  collection  of  data  through  the  use  of 
interviews  and  surveys  in  order  to  define  and  measure  the  success  of  service  contracts. 
We  utilized  a  standard  script  of  five  questions  presented  to  each  stakeholder.  The 
interviews  were  conducted  at  the  following  contracting  activities:  NAVSEA,  NAVAIR, 
and  FLC  Philadelphia.  Additionally,  interviews  were  conducted  with  CORs  associated 
with  these  contracting  activities. 

B.  INTERVIEW  QUESTIONS 

The  interview  consisted  of  three  demographic  questions  and  two  core  research 
questions: 

1 .  What  is  your  branch  of  service  or  service  affiliation? 

2.  What  is  your  current  functional  role? 

3.  What  is  your  DAWIA  level  certification? 

4.  How  do  you  define  a  successful  service  contract  and  what  factors  are 
included  in  your  definition? 

5.  How  do  you  measure  the  success  of  a  service  contract  and  what  metrics 
are  included  in  your  measurements? 

The  focus  of  the  demographic  questions  was  twofold:  to  establish  the  individual’s 
level  of  knowledge,  experience,  and  education  (DAIWA  level  certification)  and  to 
identify  the  individual’s  role  (PM,  PCO,  or  COR)  and  branch  of  service  (Navy,  Marine 
Corps,  Air  Force,  or  Army). 

The  focus  of  the  core  research  questions  was  to  determine  the  key  definitions  and 
factors  detennining  the  definition  of  a  successful  service  contract  and  to  detennine  the 
metrics  used  to  measure  a  successful  service  contract. 
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C.  PROJECT  MANAGER  FINDINGS 

In  the  course  of  our  research,  we  were  only  able  to  survey  and  interview  three 
PMs.  We  found  that  service  contracts  generally  do  not  have  a  PM  designated  and  that  the 
role  is  typically  filled  by  the  incumbent  COR.  The  definitions  and  measurements  of 
success  given  by  the  proxy  PMs  aligned  with  the  responsibilities  of  a  traditional  PM 
focusing  on  cost,  schedule,  and  performance.  We  found  that  the  PMs  factored  clarity  in 
the  statement  of  work,  effectiveness,  unproblematic,  schedule  and  tracking  costs  as  key  to 
defining  the  success  of  a  service  contract.  Further,  the  PMs  stated  that  end  user 
evaluations,  customer  satisfaction,  and  performance  (meeting  the  requirements  in  the 
statement  of  work)  were  essential  in  measuring  the  success  of  a  service  contract. 

D.  DATA  ANALYSIS  ON  PRINCIPAL  CONTRACTING  OFFICER 

1.  Overview  of  Data  Collected  on  Principal  Contracting  Officer 

Of  the  22  interviews  conducted,  16  were  with  PCOs  and  six  were  with  contract 
specialists.  The  DAWIA  level  certification  of  the  PCO  and  contract  specialists  ranged 
widely:  15  had  a  level  III  certification,  four  had  a  level  II  certification,  and  three  had  a 
level  I  certification.  Based  on  the  interviews  of  the  PCOs  and  contract  specialists,  the 
responses  for  defining  the  success  of  service  contracts  were  classified  into  nine 
categories,  discussed  in  Table  2. 


Table  2.  PCO  Success  Definitions 


SUCCESS 

CATEGORY 

COMMON  DESCRIPTORS 

Schedule 

(Outcome) 

•  Contract  is  executed  on  time 

•  Doing  it  in  a  timely  manner 

•  Meeting  milestones 

•  Getting  contracts  awarded  on  time 

•  Keep  to  the  schedule 

Unproblematic 

(Process) 

•  “Don’t  have  to  do  a  lot  of  administrative  modifications  and 
adjustments  to  the  end  contract” 
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•  “Successful  bid  process  where  there  is  not  any  animosity 
between  the  parties.” 

•  Relatively  unproblematic 

•  Runs  smoothly 

•  Operates  seamlessly 

•  “Not  hearing  from  the  customer.” 

Customer 

Satisfaction 

(Outcome) 

•  Meeting  customer  requirements 

•  Business  like  concern  for  the  customer 

•  Motivated  contractor  performance 

•  Got  what  we  paid  for 

•  Satisfies  the  mission  requirements 

•  Delivers  the  service  that  your  customer  expected 

•  “Responsive  to  the  needs  of  the  contract  and  the  needs  of 
the  activity.” 

•  Customers  are  happy. 

Well-Defined 

Requirements 

(Process) 

•  Statement  of  work  is  clearly  defined 

•  Well-defined  requirement, 

•  “Satisfies  the  test  requirements  identified  in  the 

Performance  Work  Statement.” 

•  “How  are  the  services  defined  and  is  it  defined  in  a  way  that 
it  is  clear  to  industry.” 

•  “Understanding  needs  of  the  customers.” 

Communication 

(Process) 

•  Customer  involvement 

•  Partnership  with  industry 

•  “Contractor  has  to  be  motivated  to  perform.” 

•  “Contracting  group  as  well  as  the  programs  side  all  know 
what  needs  to  get  done.” 

•  “Able  to  work  with  all  parties  involved.” 

•  “A  lot  of  integration  up  front  with  the  customer  and  our 
office.” 

•  “Mutual  understanding  as  to  what  is  required,  the  manner  in 
which  it  will  be  furnished  and  then  how  we  will  monitor 
those  services  or  the  performance,  a  good  relationship.” 
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•  “You  are  able  to  talk  and  work  things  through  and  come  up 
with  solutions.” 

Contract  Vehicle 

(Process) 

•  “Choosing  the  right  contract.” 

•  “Flexibility  in  your  contract  vehicle.” 

Efficient 

(Process) 

•  “There  would  be  some  sort  of  result  in  efficiency  that  came 
about  during  the  performance  of  that  particular  contract.” 

Adherence  to 
Regulation 

(Process) 

•  “Maintain  rules  and  regulations.” 

Maintain  Costs 

(Outcome) 

•  “Services  at  the  right  price.” 

•  “Spending  dollars  wisely.” 

•  “Keep  within  cost.” 

•  No  overruns 

•  “Contractor’s  costs  well  controlled.” 

•  “Perform  the  work  within  the  budget.” 

•  “Not  running  into  issues  where  the  contractor’s  been 
performing  but  hasn’t  had  funding.” 

Of  the  22  contracting  officers  and  specialists  interviewed,  68%  of  respondents 
specified  customer  satisfaction  and  maintaining  costs  as  a  definition  of  success.  Fifty-five 
percent  of  respondents  indicated  unproblematic  as  equivalent  definitions  of  the  success  of 
a  service  contract.  At  45%,  the  third  most  prominent  definition  of  success  was  well- 
defined  requirements.  Thirty-six  percent  of  the  respondents  indicated  communication 
relevant  to  the  success  of  a  service  contract.  Twenty-three  percent  of  the  contracting 
officers  indicated  maintaining  schedule  relevant  to  the  success  of  a  service  contract.  The 
final  three  criteria — adherence  to  regulations,  contract  vehicle,  the  efficiency — were 
specified  by  5%,  9%,  and  9%  of  the  respondents,  respectively.  PCO  definitions  of 
success  response  rates  can  be  seen  in  Figure  4. 
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Figure  4.  PCO  Success  Definitions 


2.  Overview  of  Data  Collected  on  Contracting  Officers  Metrics 

Based  on  the  interviews  of  the  contracting  officer  and  specialist,  the  responses  for 
measuring  the  success  of  service  contracts  were  classified  into  five  general  categories, 
depicted  in  Table  3. 


Table  3.  PCO  Measures  of  Success 


METRIC 

CATEGORY 

COMMON  DESCRIPTORS 

Past  Performance 

(Outcome) 

•  Contractor  Performance  Assessment  Reporting  System 
(CPARS) 

•  Past  Performance  Information  Retrieval  System 
(PPIRS) 

•  COR  annual  report,  mini  CPARS,  COR  functional 
reviews 

•  COR  verbal  periodic  review 

31 


•  COR  review,  customer  reviews 

•  Prior  contract  performance 

•  Subjective 

•  Past  history,  or  requests  from  customer  to  write  a 
contactor  a  letter  or  amend  the  contract 

•  Feedback  from  customers  on  contractor  performance 

•  Evaluation  and  recording 

•  Ability  to  get  future  contracts 

•  A  way  to  track  contractors 

Surveillance  Plan 

(Process) 

•  Is  the  contractor  performing  in  accordance  with  the 
Quality  Assurance  Surveillance  Plan  (QASP) 

•  Built  in  measurements  that  measure  quality 

•  Deficiency  reports 

•  Measurements  of  successful  performance 

•  Realistic  expectations 

•  “Performance  evaluation  factors  to  gather  from  the 
contract” 

•  “Frequency  of  when  the  service  is  performed” 

•  “Something  to  easily  put  your  eyes  on” 

•  “Performance  standard  metrics” 

•  Quality  control 

•  “Requirement  summary  of  the  various  tasks” 

•  “How  well  the  contractor  is  performing” 

•  Monitoring  turnaround  times 

Track  Costs 

(Outcome) 

•  “Burning  through  ceiling  faster  than  anticipated” 

•  “Tracking:  spend  rates,  proposed  man  hours  compared 
to  what  is  being  delivered.” 

•  “Measuring  perfonnance  to  cost  controls.” 

•  Measuring  workload  status 

•  Maintaining  good  cost  control 

•  “Being  involved  with  my  COR  early  to  say,  are  you 
looking  at  your  burn  rate?” 
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•  “Performance  is  measured  via  cost  control  tracking 
hours,  dollars,  and  parameters  to  hours  and  dollars,  not 
over  or  under  running.” 

•  “Making  sure  that  the  costs  in  the  end  are  at  the 
proposed  level  and  within  budget,  tracking  different 
modifications  to  obligations,  milestones  on  how  long 
things  are  taking.” 

Customer  Satisfaction 

(Outcome) 

•  Customer  feedback 

•  “customer  service  surveys  given  to  the  activity  for  the 
contracting  office,  immediate  feedback  on  vendor 
issues,  am  I  (the  agency)  comfortable  (uncomfortable) 
with  the  guy  (the  contractor).” 

•  “Am  I  (the  agency)  not  comfortable  with  the  guy  (the 
contractor)” 

•  “A  lot  of  times  that’s  (customer  satisfaction)  is  built 
around  personal  interaction  with  the  contractor  more  so 
than  actual  good  or  bad  performance.” 

•  “Level  of  angst,  complaints  from  discontent  customers.” 

Track  Schedule 

(Outcome) 

•  Meeting  milestones 

•  Awarded  on  time 

•  Not  a  break  in  service 

•  “Execution  of  procurement  planning  agreement” 

•  “Internal  work  in  progress  (WIP)  reports  to  track  the 
number  of  days  the  contract  has  been  in  house  or 
workload  assigned  in  the  system  with  no  errors,  timely 
delivery,  all  deliverables  received,  on  schedule,  are  the 
deliverables  on  time.” 
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Of  the  22  PCOs  and  contract  specialists  interviewed,  59%  of  the  respondents 
listed  past  performance  as  a  measurement  for  service  contract  success.  Having  a 
surveillance  plan  was  also  indicated  as  a  measurement  for  service  contract  success  by 
59%  of  respondents.  Forty-one  percent  of  the  respondents  identified  customer 
satisfaction,  27%  identified  tracking  costs,  and  23%  identified  tracking  schedule.  PCO 
measures  of  success  response  rates  are  depicted  in  Figure  5. 
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Figure  5.  PCO  Measures  of  Success 


3.  Research  Findings  from  PCO  Interviews 

Findings  for  the  PCO  produced  no  decisive  definition  of  success;  no  category 
received  an  absolute  majority  of  responses.  Several  categories  did  demonstrate  a  strong 
predilection,  with  over  half  of  the  respondents  identifying  the  categories  as  indicators  of  a 
successful  service  contract.  The  most  frequent  definitions  of  a  successful  service  contract 
were  satisfying  the  customer  and  maintaining  costs,  both  of  which  had  15  responses.  Of 
nearly  equal  significance  was  the  category  unproblematic  with  12  responses.  Well- 
defined  requirements  were  indicated  as  a  definition  of  success  by  10  respondents.  The 
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Department  of  Defense  COR  Handbook  specifies  a  properly  written  Statement  of 
Objectives  (SOO)  or  Statement  of  Work  (SOW)  as  increasing  the  likelihood  of  success 
(Director,  Defense  Procurement  and  Acquisition  Policy,  2012).  Eight  PCOs  identified 
communication  expressed  between  primarily  the  contractor,  customer,  and  PCO  as  an 
indicator  of  success.  Communication  is  a  trait  necessary  within  a  business  organization;  it 
allows  the  acquisition  team  to  work  together  to  successfully  purchase  a  service  (Garrett, 
2010).  In  comparison  with  the  FAR’s  performance  requirements  of  cost,  quality,  and 
timeliness  of  the  delivered  service,  five  of  the  respondents  indicated  schedule,  and  as 
previously  mentioned  15  indicated  maintaining  costs  as  a  definition  of  success. 
Categories  deemed  to  be  defining  factors  of  success  to  a  lesser  extent  were  contract 
vehicle,  efficiency,  and  adherence  to  regulation  with  two,  two,  and  one  responses, 
respectively. 

Similar  to  contract  success  definitions,  the  findings  yielded  no  definitive 
measurement  of  success.  Each  response  was  grouped  into  five  categories.  With 
13  responses  each,  the  two  most  frequent  responses  were  past  performance  and 
surveillance  plan.  Customer  satisfaction  yielded  nine  responses;  tracking  costs  received 
six,  and  tracking  schedule  five.  PCOs’  responses  revealed  the  measurements  of  service 
contract  success  are  in  a  majority  of  cases  very  subjective  and  ambiguous.  As  summed 
up  by  several  respondents:  “At  the  end  of  the  day,  we  don’t  have  a  numeric  score  of  yes, 
this  contractor  gets  a  B+”  and  “[we  have]  CPARS  ratings — but  a  lot  of  them  are  pretty 
subjective.”  An  analysis  of  Figure  5  reveals,  with  respect  to  outcome-based 
measurements,  past  performance  as  documented  through  the  COR  annual  reports  in 
CPARS  is  the  most  common  approach  to  measuring  contract  success. 

4.  Outcome  Versus  Process 

Our  findings  on  PCO  definitions  and  metrics  of  a  successful  service  contract 
demonstrate  both  an  outcome-  and  process-oriented  approach.  The  PCOs’  definitions  of 
a  successful  service  contract  lined  themselves  more  to  a  hybrid-oriented  approach,  with 
3  of  the  9  categories  of  responses  being  outcome  oriented  and  5  of  9  categories  process 
oriented.  The  two  most  common  response  categories  were  customer  satisfaction  and 
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maintained  costs:  both  are  outcome-based  definitions.  The  findings  illustrate  that  the 
definition  of  a  successful  service  contract  incorporates  both  outcome-  and  process-driven 
criteria.  While  the  two  prominent  definitions  of  success  were  outcome  based,  the  majority 
of  the  definitions  were  process  based.  The  majority  of  process-based  definitions  could  be 
a  result  of  the  PCOs’  greater  involvement  in  the  pre-award  phase  versus  the  post-award 
phase  of  the  service  contract  process.  The  contracting  offer  is  able  to  determine,  through 
his  pre-award  actions,  the  subsequent  results  with  respect  to  a  successful  service  contract. 

The  results  on  PCO  measurements  of  success  show  that  four  out  of  five  response 
categories  for  the  measurement  of  contract  success  were  outcome-oriented  approaches. 
The  two  significant  categories,  past  performance  and  surveillance  plan,  were  process  and 
outcome  based,  respectively.  This  mix  of  process  and  outcome  are  linked.  The 
surveillance  plan,  a  process  approach,  facilitates  the  measurement  of  past  performance, 
an  outcome  approach,  in  the  form  of  CPARS. 

Of  the  five  measurements,  we  found  past  performance,  track  costs,  and  track 
schedule  met  all  the  criteria  of  the  S.M.A.R.T.  tool.  As  previously  discussed,  the 
S.M.A.R.T.  tool  is  utilized  to  assist  corporate  managers  and  supervisors  to  determine 
quantifiable  metrics  and  objectives.  Although  surveillance  plan  accounted  for  59%  of  the 
responses,  it  failed  to  properly  address  the  criteria  of  realistic  in  S.M.A.R.T.  Therefore 
the  most  quantifiable  and  appropriate  measures  for  success  of  a  service  contract  should 
be  past  performance,  track  costs,  and  track  schedule. 


Table  4.  S.M.A.R.T.  PCO  Metrics 


CATEGORY 

S 

M 

A 

R 

T 

Past  Performance 

V 

V 

V 

V 

V 

Surveillance  Plan 

V 

V 

V 

V 

Track  Costs 

V 

V 

V 

V 

V 

Customer  Satisfaction 

V 

Track  Schedule 

V 

V 

V 

V 

V 
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E.  DATA  ANALYSIS  ON  CONTRACTING  OFFICER  REPRESENTATIVE 

1.  Overview  of  Data  Collected  on  Contractive  Officer  Representative 

Fourteen  CORs  were  interviewed  for  the  purposes  of  this  research.  DAWIA  level 
certification  was  obtained  by  35%  of  CORs  interviewed.  This  represented  a  smaller 
portion  than  we  originally  anticipated.  However,  DAWIA  certification  is  not  a 
requirement  for  appointment  to  a  COR  position. 

Based  on  the  COR  interviews,  we  arranged  the  responses  for  defining  the  success 
of  service  contracts  into  four  categories  (annotated  as  a  process  or  outcome)  with 
common  descriptions  of  each  category,  depicted  in  Table  5. 
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Table  5.  COR  Definitions  of  Success 


SUCCESS 

CATEGORY 

COMMON  DESCRIPTORS 

Schedule 

(Outcome) 

•  Timeliness,  (on)  schedule 

•  Deliverables  met  on  time. 

Performance 

(Outcome) 

•  Responsive  to  statement  of  work  (SOW) 

•  Statement  of  objectives  (SOO) 

•  Satisfactory  deliverables 

•  Performance  satisfactory  with  terms  and  conditions  of  contract 

•  Maintained  performance  within  conditions  of  contract 

•  Objectives  meet  requirements. 

Well-Defined 

Scope/Requirem 

ents 

(Process) 

•  No  deviation  in  work  perfonned 

•  Clarity  in  SOW/SOO 

•  Requirements  met  needs  of  the  end  user 

•  Clearly  written  contract 

•  Well-defined  criteria  that  are  documented 

•  Clarity  of  SOW  allows  for  proper  estimating  in  terms  of 
preparing  government  cost  estimation 

•  Specific  SOW  requirements  which  mitigate  scope  creep  by 
PM/PCO/COR. 

Communication 

(Process) 

•  Continuous  feedback  by  and  between  COR/contractor/end  user 

•  Clear  language  and  communication  between 
contractor/customer 

•  Open  communication 

Maintain  Costs 

(Outcome) 

•  Adherence  to  budget,  no  cost  overruns 

•  No  project  creep 

•  Fair  and  reasonable 

•  Cost  in  line  with  services  required 

•  Fair  cost  to  government 

•  Meets  price  guidelines 

•  Deliverables  on  time  and  within  budget 
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Of  the  14  CORs  interviewed,  71%  responded  that  well-defined  scope  and 
requirements  were  key  factors  in  defining  a  successful  service  contract,  while  58%  of 
CORs  responded  that  schedule  was  a  key  factor  in  defining  the  success  of  a  service 
contract.  Performance  had  the  third  highest  response  rate  at  50%.  The  categories  of 
communication  and  maintaining  cost  represented  21%  of  the  responses.  The  definitions 
of  success  response  rates  are  depicted  in  Figure  6. 


COR  Definitions  of  Success  (n=14) 
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Figure  6.  COR  Definitions  of  Success 


2.  Overview  of  Data  Collected  on  Contracting  Officer  Representative 
Metrics 

Based  on  the  COR  interviews,  we  categorized  the  responses  for  measuring  the 
success  of  service  contracts  into  seven  categories  (annotated  as  a  process  or  outcome) 
with  common  descriptions  of  each  category,  depicted  in  Table  6. 
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Table  6.  COR  Measures  of  Success 


METRIC 

CATEGORY 

COMMON  DESCRIPTORS 

Performance 

(Outcome) 

•  Contract  performed  in  accordance  with  terms  and 
conditions  of  the  contract 

•  Deliverables  met  quality  requirements  in  SOW 

•  Proven  response  times  on  service  contracts 

End  User  Evaluation 

(Outcome) 

•  Final  product/service  met  needs  as  of  end  user  as 
evaluated  by  end  user 

•  Summary  reviews  by  technical  experts 

Track  Schedule 

(Outcome) 

•  On  time 

•  Timeliness 

•  All  deliverables  received 

Communication 

(Process) 

•  Feedback  loop  maintained  between  contractor 

•  COR,  PM,  and/or  end  user 

•  Required  reporting  delivered/received 

No  Rework 

(Outcome) 

•  No  loss  time  on  rework 

•  No  additional  resources  utilized  for  corrections  to  end 
products/services 

Track  Costs 

(Outcome) 

•  Final  costs  remained  within  projected  costs 

•  No  cost  overruns 

•  Billable  hours  matched  contractual  levels 

•  Effectively  tracked  cost  and  relate  costs  tracking  to 
customer 

No  Protest 

(Outcome) 

•  No  contractor  protests 

•  Unproblematic  source  selection  and  award  process 
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Of  the  14  CORs  interviewed,  71%  responded  that  performance  was  the  key  factor 
in  defining  the  success  of  a  service  contract.  Track  schedule  accounted  for  28%  of 
responses.  End  user  evaluation  represented  21%.  Adherence  to  budget  represented  14% 
of  responses.  Each  of  the  following  criteria  respectively  accounted  for  7%  of  responses: 
good  communication,  no  protest,  and  no  rework.  COR  measures  of  success  response 
rates  on  depicted  in  Figure  7. 
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Figure  7.  COR  Measures  of  Success 
3.  Research  Findings  from  COR  Interviews 

The  research  findings  from  our  interviews  conducted  with  CORs  clearly 

demonstrated  varying  definitions  for  a  successful  service  contract,  as  well  as  numerous 

ways  in  which  to  measure  this  success  or  lack  thereof.  Of  the  14  CORs  interviewed,  the 

highest  response  category  was  perfonnance,  consisting  of  10  responses;  followed  by  the 

category  of  track  schedule,  with  four  responses.  The  remaining  five  categories  of 
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responses  yielded  as  high  as  three  responses  or  as  few  as  one.  Some  of  the  variability 
found  in  the  form  of  measuring  service  contracts  can  be  explained  by  the  numerous  types 
of  services  being  contracted.  These  CORs  represented  contracts  that  range  anywhere 
from  medical  supplies  to  legal  counsel.  The  heavily  diverse  nature  of  these  service 
contracts  may  have  contributed  to  the  lack  of  a  real  consistent  form  of  measurement. 

The  definitions  of  successful  service  contracts  was  less  variable  in  comparison  to 
the  responses  received  for  measuring  success  of  service  contracts.  Of  the  14  CORs 
interviewed,  10  responded  that  well-defined  scope/requirements  was  the  most  important 
criteria  for  defining  a  successful  service  contract,  while  the  categories  of  schedule  and 
performance  each  had  eight  and  seven  responses,  respectively.  Communication  and 
maintaining  costs  each  had  three  responses  each.  This  clearly  shows  that  within  the  COR 
stakeholder  group,  there  exists  a  more  standardized  definition  of  what  is  a  successful 
service  contract. 

4.  Outcome  versus  Process 

Our  findings  on  COR  definitions  and  metrics  of  a  successful  service  contract 
demonstrate  an  outcome-,  process-,  or  hybrid-oriented  approach.  The  CORs’  definition 
of  a  successful  service  contract  lined  itself  more  to  a  hybrid-oriented  approach,  with  three 
of  the  five  categories  of  responses  being  outcome  oriented  and  two  of  the  five  categories 
being  process  oriented.  The  two  most  common  response  categories  were  well-defined 
scope/requirements  (process  oriented)  and  schedule  (outcome  oriented).  The  significance 
of  this  finding  is  that  it  clarifies  the  importance  of  both  outcome-  and  process-driven 
criteria  held  by  CORs  in  determining  the  key  factors  defining  a  successful  service 
contract. 

The  CORs’  metrics  for  a  successful  service  contract  are  overwhelmingly  outcome 
driven,  with  six  of  the  seven  response  categories  being  outcome  oriented.  Additionally, 
the  two  most  prevalent  responses  were  performance  (outcome)  and  track  schedule 
(outcome).  These  findings  clearly  demonstrate  the  importance  that  outcome-specific 
measurements  have  in  the  success  of  a  service  contract  as  detennined  by  CORs. 
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When  analyzing  the  findings  on  metrics  for  a  successful  service  contract,  we 
focused  on  whether  each  category  was  found  responsive  to  the  demands  found  in  the 
S.M.A.R.T.  objectives  protocol,  as  presented  in  chapter  II 

Of  the  seven  measurements,  we  found  perfonnance,  track  schedule,  and  track 
costs  met  all  the  criteria  of  the  S.M.A.R.T.  tool.  As  previously  discussed,  the  S.M.A.R.T. 
tool  is  utilized  to  assist  corporate  managers  and  supervisors  to  determine  quantifiable 
metrics  and  objectives.  Although  end  user  evaluation  accounted  for  21%  of  the 
responses,  it  only  met  the  criteria  of  assignable  in  S.M.A.R.T.  Therefore  the  most 
quantifiable  and  appropriate  measures  for  success  of  a  service  contract  should  be 
performance,  track  schedule,  and  track  costs. 


Table  7.  Success  Measurement  Categories  and  S.M.A.R.T. 


CATEGORY 

S 

M 

A 

R 

T 

Performance 

V 

V 

V 

V 

End  User  Evaluation 

V 

Track  Schedule 

V 

V 

V 

V 

V 

Communication 

V 

No  Rework 

V 

V 

Track  Costs 

V 

V 

V 

V 

V 

No  Protest 

V 

V 

V 

We  discovered  that  the  majority  of  the  categories  for  COR  measurements  of  a 
successful  service  contract  failed  to  properly  address  the  objectives  of  S.M.A.R.T.  This 
clearly  demonstrates  that  fact  that  currently  a  substantial  portion  of  CORs  are  not 
properly  measuring  success  of  service  contracts. 
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F.  CONTRACTOR  FINDINGS 

Despite  the  emphasis  of  this  research  on  the  PCO  and  COR,  we  were  able  to 
interview  2  contractors  and  gain  their  perspective  on  the  definition  and  measurements  of 
success  in  a  service  contract.  The  contractors’  definition  of  success  includes  winning 
repeat  business,  finding  ways  to  innovate,  satisfying  the  customer,  helping  the  customer 
shape  the  SOW,  and  helping  the  government  “folks  get  up  to  speed.” 

The  contractors’  measurement  of  success  is  profit  and  the  ability  to  perform  and 
execute  the  specific  task.  Based  on  this  response,  this  measurement  was  aligned  with 
FAR  46.105:  “The  contractor  is  responsible  for  carrying  out  its  obligations  under  the 
contract”  (FAR,  2012).  Success  is  also  measured  in  terms  of  spend  rates — that  is,  over  or 
under  budget  (how  many  people  are  put  to  work  and  how  many  people  have  to  be  hired 
to  complete  the  task).  The  definitions  and  measurements  of  success  allude  to  maximizing 
profit  as  the  overarching  goal  of  a  contractor. 

G.  CONTRACITNG  OFFICER  REPRESENTATIVE  AND  PRINCIPAL 

CONTRACTING  OFFICER  COMPARISON 

1.  Contracting  Officer  Representative  and  Principal  Contracting  Officer 

Similarities  in  Definitions  of  Success 

Our  research  revealed  the  following  similarities  between  CORs  and  PCOs  when 
defining  the  success  of  a  service  contract: 

•  Schedule 

•  Maintaining  costs 

•  Communication 

•  Well-defined  scope  and  requirements. 

a.  Schedule 

The  data  shows  both  the  PCO  and  COR  identify  maintaining  schedule  as  a 
common  definition  of  success.  PCOs  and  CORs  are  both  members  of  the  acquisition 
team  guided  by  the  objectives  and  regulations  of  the  FAR.  In  service  contracts,  the  COR 
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often  performs  the  functions  of  a  PM;  thus,  schedule  is  an  important  factor  to  the  COR  as 
well  as  progressing  the  contract  through  the  acquisition  life  cycle.  Additionally,  the  FAR 
states  the  acquisition  team  must  satisfy  the  customer  in  terms  of  cost,  quality  and 
timeliness  of  the  delivered  goods  or  services  (FAR,  2012).  Furthermore,  the  COR 
handbook  states  that  the  COR  should  ensure  that  the  contract  for  goods  or  services  is 
timely  and  highlights  schedule  as  a  key  assessment  factor  (Director,  Defense 
Procurement  and  Acquisition  Policy,  2012).  The  schedule  is  the  most  visible  and 
scrutinized  perfonnance  measure  for  both  the  PCO  and  the  COR.  For  example,  the 
workload  of  the  PCO  is  driven  by  their  WIP  report,  which  tracks  the  number  of  days  the 
contract  has  been  assigned  in  the  system.  One  of  the  key  assessment  factors  of  the  COR 
is  schedule  (Director,  Defense  Procurement  and  Acquisition  Policy,  2012). 

b.  Maintain  Costs 

Sixty-eight  percent  of  PCOs  and  21%  of  CORs  defined  maintaining  costs 
as  a  definition  of  success.  Given  the  budget  constraints,  DoD  acquisitions  are  limited  by 
scarce  resources.  As  such,  maintaining  costs  is  a  high  priority.  Failure  to  maintain 
budget  could  result  in  a  possible  violation  of  the  Anti-Deficiency  Act,  defined  as  an 
obligation  in  excess  of  available  funds.  As  mentioned  earlier,  the  COR  often  assumes  the 
responsibilities  of  the  PM  on  service  contracts;  therefore,  cost  becomes  a  responsibility. 
The  data  shows  a  difference  of  47%  between  PCO  and  COR  responses.  The  disparity  is  a 
result  of  the  PCOs’  accountability  for  contract  administration;  therefore,  the  PCO  places 
a  higher  emphasis  on  maintaining  costs.  This  is  supported  by  the  FAR  (2012),  which 
states  that  PCOs  “shall  ensure  that  the  requirements  of  1.602- 1(b)  have  been  met  and  that 
sufficient  funds  are  available  for  obligation.” 

c.  Communication 

The  data  concludes  that  CORs  and  PCOs  share  communication  as  an 
attribute  definitive  to  success.  Common  to  both,  the  FAR  states  the  federal  acquisition 
system  will  foster  cooperative  relationships  between  the  government  and  its  contractors. 
More  importantly  in  each  case,  successful  communication  and  continuous  feedback 
between  PCO  and  COR  and  acquisition  team  members  contributes  to  a  service  contract 
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success.  When  all  members  of  the  project  team  communicate  properly,  clear  and  concise 
objectives  are  conveyed  to  the  team  and  the  goal  is  understood  and  identified. 
Additionally,  clear  communication  is  fundamental  to  the  resolution  of  deficiencies  in 
service  contracts. 

d.  Well-defined  Scope  and  Requirements 

The  data  shows  71%  of  the  CORs  versus  45%  of  the  PCOs  define  success 
as  a  well-defined  scope  and  requirements.  The  COR  and  the  PCO  are  focused  on 
meeting  the  activity’s  need  and  satisfying  all  technical  aspects  of  the  SOW.  A  well- 
defined  scope  and  set  of  requirements  lay  the  foundation  the  required  services. 
Therefore,  the  PCO  and  the  COR  have  a  vested  interest  in  having  a  well-defined  scope 
and  meeting  requirements  in  the  SOW.  With  a  proper  SOW,  contract  performance  is 
more  likely  to  be  successful.  One  of  the  key  pre-award  duties  in  which  the  COR  may  be 
involved  is  documenting  requirements.  The  requirements  package  is  critical  to  the 
success  of  an  acquisition  because  it  commits  the  funds  and  establishes  the  basis  for  a 
contractual  action. 

2.  Contracting  Officer  Representative  and  Principal  Contracting  Officer 

Differences  in  Definitions  of  Success 

Our  research  revealed  the  following  differences  between  CORs  and  PCOs  when 
defining  the  success  of  a  service  contract: 

•  unproblematic, 

•  customer  satisfaction, 

•  adherence  to  rules  and  regulations, 

•  contract  vehicle,  and 

•  efficiency. 
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a. 


Unproblematic 


Fifty-five  percent  of  the  PCOs  stated  unproblematic  as  a  definition  of 
success.  The  PCO  is  responsible  for  administrating  all  aspects  of  the  contract;  therefore, 
a  problem-free  contract  reduces  the  workload  of  additional  modifications  and  contract- 
related  issues.  Increases  in  workload  are  even  more  substantial  given  the  inadequate  size 
of  the  acquisition  workforce.  Contrarily,  CORs  are  less  concerned  with  contract 
administration  and  are  more  focused  on  technical  evaluation  factors.  The  COR  is  more 
concerned  with  the  surveillance  plan  and  delivering  problems  to  the  contractor.  Problems 
are  not  a  true  concern  to  the  COR.  According  to  the  COR  handbook,  the  COR  is 
responsible  for  bringing  any  issues  or  performance  problems  to  the  attention  of  the  PCO. 

b.  Customer  Satisfaction 

Sixty-eight  percent  of  the  PCOs  conveyed  customer  satisfaction  as  a 
definition  of  success,  demonstrating  the  emphasis  PCOs  place  on  customer  satisfaction. 
The  PCO  is  more  customer  service  oriented  in  comparison  with  the  COR,  whose  primary 
purpose  is  to  monitor  the  perfonnance  of  the  contract.  A  PCO  must  evaluate  all  relevant 
surrounding  circumstances  to  discern  the  customers’  expectations  (Hirsch,  1986). 
Additionally,  customer  satisfaction  plays  a  central  role  in  determining  contractors’  past 
performance,  an  evaluation  factor  required  by  the  FAR  that  results  in  contract  renewal  or 
follow-on  contracts. 


c.  Adherence  to  Rules  and  Regulations 

Five  percent  of  the  PCOs  stated  adherence  to  rules  and  regulations  was  a 

definition  of  success.  In  accordance  with  the  FAR  the  PCO  ensures  all  requirements  of 

law,  executive  orders,  regulations,  and  all  other  applicable  procedures,  including 

clearances  and  approvals,  are  adhered  to  in  government  contracting.  The  COR,  while  a 

representative  of  the  PCO,  has  no  true  authority  to  affect  change  such  as  alter  the  terms 

and  conditions  of  the  contract;  therefore,  the  COR  places  no  emphasis  on  compliance  to 

rules  and  regulations.  The  COR’s  only  liability  is  in  the  form  of  unauthorized  acts. 

Adherence  to  rules  and  regulations  is  a  core  function  and  inherent  to  the  position  of  PCO; 

therefore,  adherence  to  rules  and  regulations  is  implied. 
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d.  Contract  Vehicle 

Nine  percent  of  the  PCOs  specified  the  contract  vehicle  as  definitive  of 
success.  The  CORs  gave  no  indication  to  contract  vehicle  as  a  definition  of  success.  This 
is  owing  to  the  possible  lack  of  integration  of  the  COR  in  the  pre-award  phase  of  service 
contract  process.  The  Department  of  Defense  COR  Handbook  (Director,  Defense 
Procurement  and  Acquisition  Policy,  2012)  only  lists  pre-award  activities  the  COR  may 
be  involved  with  and  makes  no  reference  to  required  involvement. 

e.  Efficiency 

Nine  percent  of  PCOs  interviewed  specified  efficiency  as  a  definition  of 
success.  The  CORs  made  no  acknowledgment  of  efficiency  as  a  definition  of  service 
contract  success.  As  previously  determined,  the  CORs’  definition  of  success  is  primarily 
outcome  oriented.  The  COR  is  concerned  with  tracking  the  contract’s  compliance 
according  to  its  terms  and  conditions  and  delivery  of  the  final  product.  In  contrast,  the 
PCO  definition  of  success  is  primarily  process  driven  and  processes  lend  themselves  to 
continuing  improvement.  As  a  PCO  explained,  contract  efficiency  leads  to  a  reduction  in 
contract  risk,  allowing  the  government  to  switch  contract  vehicles  from  a  risk-burdened 
cost  contract  to  a  fixed  contract,  which  transfers  risk  to  the  contractor. 

3.  Contracting  Officer  Representative  and  Principal  Contracting  Officer 

Similarities  in  Measurements  of  Success 

Our  research  revealed  the  following  similarities  between  CORs  and  PCOs  when 
measuring  the  success  of  a  service  contract: 

•  past  performance  and  performance, 

•  track  costs, 

•  track  schedule,  and 

•  customer  satisfaction  and  end  user  evaluation 
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a.  Past  Performance  and  Performance 

Data  shows  PCOs  and  CORs  measure  past  perfonnance  and  perfonnance 
59%  and  71%,  respectively.  PCOs’  and  CORs’  measures  of  performance  are  both 
outcome -based  measurements.  Both  measures  employ  user  feedback  as  a  gauge  of 
success.  The  COR  measures  performance  according  to  the  surveillance  plan,  which 
provides  input  to  the  PCO’s  measurement  of  past  performance.  Based  on  the  research, 
performance  and  past  performance  are  two  of  the  most  relevant  tools  used  for  measuring 
success. 


b.  Track  Costs 

Data  revealed  PCOs  and  CORs  measure  tracking  costs  and  adherence  to 
budget  27%  and  14%,  respectively.  This  relates  directly  to  the  responsibilities  of  both  the 
PCO  and  the  COR.  As  outlined  in  the  COR  handbook,  the  COR  must  conduct 
surveillance  to  maintain  costs  (Director,  Defense  Procurement  and  Acquisition  Policy, 
2012).  The  FAR  specifically  mandates  that  PCOs  ensure  adhering  to  procurement  laws 
and  regulations  such  as  the  Anti-Deficiency  Act  (Federal  Acquisition  Regulation,  2012). 

c.  Track  Schedule 

We  found  that  the  CORs  and  the  PCOs  placed  similar  emphasis  on 
tracking  schedule,  with  response  rates  of  28%  and  23%,  respectively.  We  found  that  the 
response  rates  for  tracking  sechdule  were  relatively  low  given  the  the  requirements  of 
both  the  COR  and  PCO,  in  accordance  with  the  COR  handbook,  to  ensure  that  contract 
performance  is  timely  and  within  scope  of  the  work  (Director,  Defense  Procurement  and 
Acquisition  Policy,  2012).  Additionally,  the  FAR  states  that  the  COR  and  PCO  must 
satisfy  the  customer  in  terms  of  timeliness  of  the  delivered  service  (FAR,  2012). 

d.  Customer  Satisfaction  and  COR  End  User  Evaluation 

Data  revealed  the  PCOs  placed  a  higher  importance  on  customer 
satisfaction  than  the  CORs;  the  two  groups  had  response  rates  of  41%  and  21%, 
respectively.  This  demonstrates  that  the  PCO  places  more  of  an  emphasis  on  customer 
satisfaction  as  a  measure  of  success  than  the  COR.  The  low  response  rate  from  CORs 
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was  surprising  given  the  close  interaction  of  CORs  with  end  users  and  technical 
evaluators  within  services  contracts.  However,  CORs  are  typically  nominated  for  their 
technical  expertise  and  not  their  customer  relationship  skills.  Conversely,  PCOs’ 
positions  are  the  customer  service  oriented.  Additionally,  in  certain  cases  the  COR  is 
also  the  customer. 

4.  Contracting  Officer  Representative  and  Principal  Contracting  Officer 

Differences  in  Measurements  of  Success 

Our  research  revealed  the  following  differences  between  CORs  and  PCOs  when 
measuring  the  success  of  a  service  contract: 

•  no  rework,  no  protest,  and  good  communication;  and 

•  PCO  surveillance  plan 

a.  No  Rework,  No  Protest,  and  Good  Communication 

CORs  listed  no  rework,  no  protests,  and  good  communication  as  measures 
of  a  successful  service  contract.  However,  it  is  prudent  to  note  that  the  response  rate  for 
all  three  categories  was  only  7%  each,  equating  to  only  one  responder  in  each  category. 
As  such,  these  categories  represent  a  less  than  significant  metric  for  success  in  service 
contracts. 


b.  PCO  Surveillance  Plan 

The  PCO’s  surveillance  plan  represented  one  of  the  highest  response  rates 
at  59%  (13  respondents  out  of  22).  The  FAR  states  that  government  contract  quality 
assurance  shall  be  perfonned  at  such  times  and  places  as  may  be  necessary  to  determine 
that  the  services  confonn  to  contract  requirements  (FAR,  2012).  The  COR  handbook 
clearly  states  that  the  QASP  is  an  important  tool  for  assessing  the  service  contract  for  the 
COR  (Director,  Defense  Procurement  and  Acquisition  Policy,  2012).  However,  the 
CORs  exhibited  zero  responses  for  surveillance  plan,  demonstrating  a  possible  lack  of 
training  or  that  CORs’  experience  involves  contracts  below  the  simplified  threshold. 
Further,  the  QASP  was  prepared  by  the  contractor.  According  to  the  FAR,  “The 

Government  may  either  prepare  the  quality  assurance  surveillance  plan  or  require  the 
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offerors  to  submit  a  proposed  quality  assurance  surveillance  plan  for  the  Government’s 
consideration  in  development  of  the  Government’s  plan”  (FAR,  2012). 


H.  CONCLUSIONS 

Our  findings  corroborate  the  deficiency  found  within  the  GAO  report  on  best 
practices  in  acquisition  services,  that  there  are  few  services  contracting-related  annual 
performance  metrics  (GAO,  2002). 

Further,  we  found  that  there  is  no  standardized  definition  or  measurements  for 
success  of  service  contracts.  While  similarities  do  exist  between  the  definitions  and 
measures  of  success  of  a  service  contract,  the  levels  of  emphasis  placed  on  those  similar 
categories  in  many  cases  were  disproportionate. 

In  addition,  those  metrics  in  place  were  found  to  lack  both  in  terms  of  their  ability 
to  be  quantified  and  their  ability  to  meet  the  requirements  of  S.M.A.R.T.  measurements. 

We  discovered  that  the  differing  objectives  and  duties  of  the  stakeholders  clearly 
affected  the  factors  that  each  stakeholder  emphasized  when  defining  and  measuring  the 
success  of  service  contracts.  Stakeholder  theory  identifies  the  conflicts  that  arise  between 
the  acquisition  team  and  how  these  conflicts  can  lead  to  this  lack  of  uniformity. 

In  our  research,  we  identified  a  clear  lack  of  establishment  of  PMs  within  DoD 
services  contracting.  In  many  cases,  the  incumbent  COR  was  the  PM.  This  supports 
GAO  findings  that  the  DoD  lacks  the  proper  management  structure  and  processes  for  the 
managing  services  contracts  (GAO,  2002b,  2007b;  DoD  IG,  2009) 

Finally,  we  discovered  that  no  uniform  certification  process  was  established  and 
required  for  CORs  within  services  contracting  at  the  time  of  our  research.  This  validates 
the  GAO  report’s  conclusions  on  defense  acquisition  workforce  training,  stating  that  a 
lack  of  training  for  defense  acquisition  workforce  personnel  continues  to  plague  DoD 
services  contracting  efforts  (GAO,  2002).  To  our  knowledge,  reform  measures  are  in 
process  for  the  COR  training  certification  process. 

We  discovered  the  COR  is  not  as  involved  in  the  pre-award  phase  of  service 
contracts.  According  to  the  COR  handbook,  the  COR  is  responsible  for  preparing  the 
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SOW/PWS  and  surveillance  plan.  Our  COR  data  indicates  no  reference  to  surveillance 
as  a  measure  of  success.  Typically,  the  person  who  develops  the  SOW/PWS  also 
develops  the  surveillance  methods.  The  CORs’  lack  of  reference  to  a  surveillance  plan 
implies  no  significant  involvement  in  developing  the  SOW/PWS. 

We  found  that  the  majority  of  PCOs  put  a  large  emphasis  on  past  performance 
and  surveillance  plans,  yet  QASP  is  only  mandatory  for  use  in  acquisitions  in  excess  of 
simplified  acquisition  threshold.  Additionally,  CPARS  is  only  mandated  for  services 
acquisitions  in  excess  of  $1  million.  Yet,  over  83%  of  United  States  federal  acquisitions 
have  a  dollar  value  on  average  of  $25,000.  Therefore,  the  majority  of  acquisitions  are  not 
required  to  possess  any  surveillance  plan,  nor  do  they  require  any  evaluation  on  past 
performance.  Only  1%  of  federal  acquisition  contracts  are  over  $1  million  (Garrett, 
2011). 

I.  RECOMMENDATIONS 

A  standardized  definition  of  a  successful  service  contract  must  be  implemented, 
incorporating  the  proper  factors  with  the  correct  level  of  emphasis.  Additionally, 
standardized  and  quantifiable  measures  must  be  implemented  within  DoD  services 
contracting.  These  measures  should  align  with  S.M.A.R.T.  metrics  and  incorporate  a 
more  objective  orientation.  This  standard  should  incorporate  only  those  definitions  and 
measures  that  both  support  the  objectives  and  goals  of  the  DoD,  as  outlined  in  chapters  I 
and  II,  and  are  validated  by  correlation  in  responses  between  stakeholders.  Furthermore, 
the  standard  metrics  incorporated  should  meet  the  criteria  as  outlined  by  S.M.A.R.T.  We 
suggest  utilizing  a  balanced  outcome  and  process  oriented  approach,  incorporating  the 
following  categories  when  defining  a  successful  service  contract: 

•  schedule,  maintain  costs,  communication  and  well  defined  requirements 

The  standard  measure  for  a  successful  service  contract  should  incorporate  solely 
an  outcome  driven  approach  utilizing  the  following  three  metrics: 

•  performance,  track  costs,  and  track  schedule 
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These  standard  metrics  are  aligned  with  the  FAR’s  detennination  that  the 
customer  should  be  satisfied  in  tenns  of  cost,  timeliness,  and  quality  of  the  delivered 
product  or  service.  Additionally,  in  accordance  with  the  COR  handbook,  the  COR  and 
PCO  must  ensure  that  contract  performance  is  timely  and  within  scope  of  the  work 
(Director,  Defense  Procurement  and  Acquisition  Policy,  2012).  Our  recommended 
standards  of  defining  and  measuring  the  success  of  a  service  contact  ensure  adherence  to 
these  guidelines.  These  standards  definitions  and  metrics  for  a  successful  service  contract 
align  with  the  goals  and  initiatives  as  outlined  by  the  USD  (AT&L),  to  properly  manage 
and  assess  each  service  contract  to  determine  its  performance.  Incorporation  of  these 
standards  will  allow  for  a  more  uniform  analysis  of  this  performance  and  could  lead 
directly  to  the  goals  and  objectives  of  the  USD  (AT&L),  to  promote  efficient  and 
effective  services  contracting,  while  reducing  waste  and  producing  savings.  Additionally, 
this  standard  set  of  metrics  and  definitions  will  lead  to  better  analysis  on  performance  of 
a  service  contractor,  leading  to  a  selection  of  more  efficiency  and  effective  contractors  in 
future  programs. 

In  addition,  these  standards  for  defining  and  measuring  a  successful  service 
contract  could  be  represented  using  a  scorecard  approach.  The  balanced  scorecard  is  a 
planning  and  management  tool  within  government  organizations  to  align  activities  to  the 
goals  and  objectives  of  the  organization,  improve  communications,  and  monitor 
organization  perfonnance  against  goals.  The  scorecard  employs  a  performance 
measurement  framework  added  to  financial  metrics  to  give  managers  a  complete  view  of 
organizational  performance  (Monczka,  Handfield,  Giunipero,  &  Patterson,  2011). 
Although  important,  cost  is  a  lagging  indicator  of  performance  and  not  the  only 
determining  factor.  The  scorecard  ensures  other  related  factors  (tracking  schedule  and 
performance)  are  given  proper  emphasis  in  evaluating  service  contract  success. 

Incorporating  these  standardized  definitions  and  quantifiable  measurements  in  the 
form  of  a  scorecard  will  mitigate  the  conflicts  that  arise  due  to  the  differing  objectives 
and  goals  of  the  various  stakeholders  within  DoD  services  contracting. 

The  PM  is  an  integral  member  of  the  IPT  and  an  important  stakeholder  in  the 

contracting  process.  Every  service  contract  should  be  evaluated  on  complexity  and 
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recognize  when  a  need  exists  to  incorporate  a  properly  trained  and  assignable  PM  as  a 
key  stakeholder  (Phillips,  2007).  PMs  provide  strong  communication  skills  and 
leadership  to  the  entire  team. 

An  improved  application  of  surveillance  and  past  performance  could  be 
implemented.  For  example,  the  FAR  should  require  a  QASP  for  contracts  below  the 
simplified  acquisition  threshold.  Incorporating  a  QASP  will  ensure  oversight  for  the 
majority  of  service  contracts.  Even  though  the  FAR  15.304  states  that  past  perfonnance 
evaluation  should  always  be  evaluated  for  negotiated  competitive  acquisitions  expected 
to  exceed  the  simplified  acquisition  threshold,  CPARS  is  only  required  for  acquisitions 
above  the  $1  million  threshold.  The  CPARS  threshold  should  be  amended  to  include 
contracts  below  $  1  million  down  to  a  minimum  of  the  simplified  acquisition  threshold  or 
at  a  point  acceptable  to  capture  the  majority  of  service  contract  past  performance  data.  In 
order  to  emphasize  COR  importance,  individual  contracting  agencies  should  include 
instructions  and  directives  mandating  involvement  of  the  COR  in  the  pre-award  phase 
activities,  such  as  development  of  the  SOW/PWS. 

J.  SUMMARY 

In  this  chapter  we  presented  and  analyzed  the  data  we  collected  from  the  research 
to  answer  the  three  research  questions: 

1.  How  are  successful  service  contracts  within  the  DoD  being  defined  by 
different  stakeholders? 

2.  How  are  service  contracts  being  measured  within  the  DoD  by  different 
stakeholders? 

3.  How  should  service  contracts  be  defined  and  measured  within  theDoD? 

We  presented  conclusions  and  recommendations  based  on  our  analysis.  In  the 
next  chapter,  we  summarize  our  findings  and  provide  recommendations  for  further 
research. 
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V.  SUMMARY,  CONCLUSION,  AND  AREAS  FOR 
FURTHER  RESEARCH 


A.  SUMMARY 

Over  the  last  few  decades,  a  growing  trend  of  increased  dollar  value  in  service 
contracts  for  Department  of  Defense  (DoD)  contracting  has  become  apparent.  Relative  to 
supply  contracts,  services  acquisition  has  continued  to  grow  both  in  terms  of  dollar  value 
and  in  range  of  acquisitions.  Contract  obligations  rose  to  over  $387  billion  in  2008,  with 
nearly  $200  billion  spent  on  services  alone  (Hutton  &  Solis,  2009).  Funding  spent  on 
service  contracts  grew  steadily  from  1990-2010,  constituting  roughly  42%  of  the  total 
spending  on  contracts  by  the  DoD,  exclusive  of  research  and  development  services 
contracts.  As  such,  a  standardized  approach  to  defining  and  measuring  the  success  of 
services  contracts  is  essential. 

The  DoD  IG  and  GAO  have  indicated  poorly  defined  requirements  and 
inadequate  requirements  management  as  a  problem  in  service  contracting  (GAO,  2007b; 
DoD  IG  2009)  The  GAO  has  described  the  DoD’s  current  approach  to  service  contract 
management  as  reactive  and  not  fully  addressing  the  key  factors  of  success  (GAO, 
2007b).  DoD  IG  and  GAO  reports  have  consistently  identified  issues  in  service  contract 
administration  and  oversight  (GAO,  2005,  2007a,  2007b;  DoD  IG,  2009).  According  to 
the  GAO,  the  poor  management  of  service  contracts  has  undermined  the  government’s 
ability  to  obtain  a  good  value  for  the  money  spent  and  contributed  to  the  GAO’s  decision 
to  designate  contract  management  a  high-risk  area  for  the  DoD  (GAO,  2001,  2007c, 
2011). 

The  DoD  implements  cross  functional  teams  through  the  use  of  an  Integrated 
Product  Team  (IPT).  It  is  useful  to  note  that  the  IPT  is  primarily  used  in  contracting  for 
products  to  facilitate  the  process  of  meeting  cost,  perfonnance,  and  schedule  objectives 
from  product  concept  through  production,  including  field  support  (Press,  2012).  In 
service  contracting  the  IPT  is  a  team  composed  of  representatives  from  appropriate 
functional  disciplines  working  together  to  identify  and  resolve  issues;  make  sound,  timely 
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recommendations  in  an  effort  to  facilitate  decision-making;  and  build  successful 
programs  that  meet  the  warfighter’s  needs  (Press,  2012). 

With  the  differing  goals,  objectives,  and  responsibilities  of  the  various 
stakeholders  (PM,  PCO,  COR)  within  DoD  services  contracting,  conflicts  arise  with 
regard  to  how  a  successful  service  contract  is  defined  and  measured  between  the 
stakeholders.  This  conflict  is  explained  and  understood  through  stakeholder  theory. 

B.  CONCLUSIONS 

1.  Findings 

The  research  answered  the  questions  contained  in  Chapter  I: 

1 .  How  are  successful  service  contracts  within  the  DoD  defined  by  different 
stakeholders  and  what  factors  are  considered  in  their  definitions? 

2.  How  are  service  contracts  being  measured  within  the  DoD  by  different 
stakeholders? 

3.  How  should  service  contracts  be  defined  and  measured  within  the  DoD? 

We  found  that  there  is  no  standardized  definition  or  measurements  for  success  of 
service  contracts.  While  similarities  do  exist  between  the  definitions  and  measures  of 
success  of  a  service  contract,  the  weighted  value  of  each  factor  and  metric  varies  between 
the  stakeholders.  Many  of  the  metrics  that  were  established  failed  to  properly  address  the 
characteristics  of  S.M.A.R.T.  either  partially  or  entirely. 

We  discovered  that  the  differing  objectives  and  roles  of  each  stakeholder  clearly 
affected  the  factors  that  each  considered  when  defining  and  measuring  the  success  of 
service  contracts.  Our  research  revealed  that  CORs  and  PCOs  define  the  factors  of  a 
successful  service  contract  as  the  following:  schedule,  maintain  costs,  communication, 
and  well  defined  requirements.  Additionally,  we  discovered  CORs  and  PCOs  similarly 
measure  a  successful  service  contract  as  the  following:  perfonnance,  track  costs, 
customer  satisfaction  and  end  user  evaluation  and  track  schedule. 
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Further,  we  discovered  a  lack  of  properly  established  PMs  within  DoD  services 
contracting,  with  contributes  to  the  deficiencies  already  present  in  services  contracting. 
Additionally,  we  discovered  that  no  standardized  certification  process  was  established 
and  required  for  CORs  within  services  contracting. 

We  also  discovered  the  COR  is  not  always  involved  in  the  pre-award  phase  of 
service  contracts  and  current  application  of  QASP  and  CPARS  is  inadequate. 

2.  Recommendations 

A  standardized  definition  of  a  successful  service  contract  must  be  implemented, 
incorporating  the  proper  factors  with  the  correct  level  of  emphasis.  Additionally, 
standardized  and  quantifiable  measures  must  be  implemented  within  DoD  services 
contracting.  These  measures  should  align  with  S.M.A.R.T.  metrics  and  incorporate  a 
more  objective  orientation  than  currently  that  which  currently  exists.  Incorporating  these 
standardized  definitions  and  quantifiable  measurements  will  mitigate  the  conflicts  that 
arise  due  to  the  differing  objectives  and  goals  of  the  various  stakeholders  within  DoD 
services  contracting.  Every  service  contract  should  be  evaluated  on  complexity  and 
evaluators  should  recognize  when  a  need  exists  for  a  PM.  An  improved  application  of 
surveillance  and  past  performance  should  be  implemented.  In  order  to  emphasize  COR 
importance,  individual  contracting  agencies  should  include  instructions  and  directives 
mandating  involvement  of  the  COR  in  the  pre-award  phase  activities,  such  as 
development  of  the  SOW/PWS. 

In  addition,  we  suggest  utilizing  a  balanced  outcome  and  process  oriented 
approach,  incorporating  the  following  categories  when  defining  a  successful  service 
contract:  schedule,  maintain  costs,  communication  and  well  defined  requirements.  The 
standard  measure  for  a  successful  service  contract  should  incorporate  solely  an  outcome 
driven  approach  utilizing  the  following  three  metrics:  perfonnance,  track  costs,  and  track 
schedule.  These  standards  for  defining  and  measuring  a  successful  service  contract  could 
be  represented  using  a  balanced  scorecard  approach.  The  balanced  scorecard  is  a 
planning  and  management  tool  within  government  organizations  to  align  activities  to  the 
goals  and  objectives  of  the  organization,  improve  communications,  and  monitor 
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organization  performance  against  goals.  Incorporating  these  standardized  definitions  and 
quantifiable  measurements  in  the  form  of  a  scorecard  will  mitigate  the  conflicts  that  arise 
due  to  the  differing  objectives  and  goals  of  the  various  stakeholders  within  DoD  services 
contracting. 

C.  AREAS  FOR  FURTHER  RESEARCH 

Our  research  participants  consisted  solely  of  commands  associated  with  the 
United  Sates  Navy.  As  such,  we  recommend  continuing  this  research  into  the  various 
contracting  commands  within  the  United  States  Air  Force,  Anny,  and  Marine  Corps.  We 
were  also  able  to  incorporate  very  little  data  from  PMs  and  contractors  within  DoD 
services  contracting.  We  recommend  that  future  research  incorporate  a  larger  portion  of 
PMs  and  contractors.  Additionally,  we  recommend  that  future  research  incorporate 
greater  use  of  the  survey  tools,  such  as  Survey  Money,  due  to  its  ability  to  reach  a  larger 
audience  more  effectively  and  efficiently. 
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